Repository: CS-SI/eodag
Pull request #212 (2021-03-26 13:37:31)

v2.2.0


Patch:
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
new file mode 100644
index 00000000..f075d063
--- /dev/null
+++ b/.github/pull_request_template.md
@@ -0,0 +1,14 @@
+### Your checklist for this pull request
+Please review the [guidelines for contributing](../CONTRIBUTING.md) to this repository.
+
+- [ ] Make sure you are making a pull request against the **develop branch** (left side). Also you should start *your branch* off *our develop*.
+- [ ] Check the commit's or even all commits' message styles matches our requested structure.
+- [ ] Check your code additions will fail neither code linting checks nor unit test.
+
+### Description
+Please describe your pull request.
+
+### Further comments
+If this is a relatively large or complex change, kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc...
+
+Thank you!
diff --git a/CHANGES.rst b/CHANGES.rst
index 59bcdfc1..531c36c3 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -1,6 +1,19 @@
 Release history
 ---------------
 
+2.2.0 (2021-03-26)
+++++++++++++++++++
+
+- New ``search_all`` and ``search_iter_page`` methods to simplify pagination handling
+  (`GH-190 <https://github.com/CS-SI/eodag/pull/190>`_)
+- Docker-compose files for STAC API server with STAC-browser (`GH-183 <https://github.com/CS-SI/eodag/pull/183>`_)
+- Fixed USGS plugin which now uses M2M API (`GH-209 <https://github.com/CS-SI/eodag/pull/209>`_)
+- Windows support added in Continuous Integration (`GH-192 <https://github.com/CS-SI/eodag/pull/192>`_)
+- Fixes issue with automatically load configution from EODAG external plugins, fixes
+  `GH-184 <https://github.com/CS-SI/eodag/issues/184>`_
+- More explicit signature for ``setup_logging``, fixes `GH-197 <https://github.com/CS-SI/eodag/issues/197>`_
+- Various minor fixes
+
 2.1.1 (2021-03-18)
 ++++++++++++++++++
 
diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md
new file mode 100644
index 00000000..957e47c8
--- /dev/null
+++ b/CODE_OF_CONDUCT.md
@@ -0,0 +1,133 @@
+
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+
+We as members, contributors, and leaders pledge to make participation in our
+community a harassment-free experience for everyone, regardless of age, body
+size, visible or invisible disability, ethnicity, sex characteristics, gender
+identity and expression, level of experience, education, socio-economic status,
+nationality, personal appearance, race, caste, color, religion, or sexual identity
+and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming,
+diverse, inclusive, and healthy community.
+
+## Our Standards
+
+Examples of behavior that contributes to a positive environment for our
+community include:
+
+* Demonstrating empathy and kindness toward other people
+* Being respectful of differing opinions, viewpoints, and experiences
+* Giving and gracefully accepting constructive feedback
+* Accepting responsibility and apologizing to those affected by our mistakes,
+  and learning from the experience
+* Focusing on what is best not just for us as individuals, but for the
+  overall community
+
+Examples of unacceptable behavior include:
+
+* The use of sexualized language or imagery, and sexual attention or
+  advances of any kind
+* Trolling, insulting or derogatory comments, and personal or political attacks
+* Public or private harassment
+* Publishing others' private information, such as a physical or email
+  address, without their explicit permission
+* Other conduct which could reasonably be considered inappropriate in a
+  professional setting
+
+## Enforcement Responsibilities
+
+Community leaders are responsible for clarifying and enforcing our standards of
+acceptable behavior and will take appropriate and fair corrective action in
+response to any behavior that they deem inappropriate, threatening, offensive,
+or harmful.
+
+Community leaders have the right and responsibility to remove, edit, or reject
+comments, commits, code, wiki edits, issues, and other contributions that are
+not aligned to this Code of Conduct, and will communicate reasons for moderation
+decisions when appropriate.
+
+## Scope
+
+This Code of Conduct applies within all community spaces, and also applies when
+an individual is officially representing the community in public spaces.
+Examples of representing our community include using an official e-mail address,
+posting via an official social media account, or acting as an appointed
+representative at an online or offline event.
+
+## Enforcement
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported to the community leaders responsible for enforcement at
+admin@geostorm.eu.
+All complaints will be reviewed and investigated promptly and fairly.
+
+All community leaders are obligated to respect the privacy and security of the
+reporter of any incident.
+
+## Enforcement Guidelines
+
+Community leaders will follow these Community Impact Guidelines in determining
+the consequences for any action they deem in violation of this Code of Conduct:
+
+### 1. Correction
+
+**Community Impact**: Use of inappropriate language or other behavior deemed
+unprofessional or unwelcome in the community.
+
+**Consequence**: A private, written warning from community leaders, providing
+clarity around the nature of the violation and an explanation of why the
+behavior was inappropriate. A public apology may be requested.
+
+### 2. Warning
+
+**Community Impact**: A violation through a single incident or series
+of actions.
+
+**Consequence**: A warning with consequences for continued behavior. No
+interaction with the people involved, including unsolicited interaction with
+those enforcing the Code of Conduct, for a specified period of time. This
+includes avoiding interactions in community spaces as well as external channels
+like social media. Violating these terms may lead to a temporary or
+permanent ban.
+
+### 3. Temporary Ban
+
+**Community Impact**: A serious violation of community standards, including
+sustained inappropriate behavior.
+
+**Consequence**: A temporary ban from any sort of interaction or public
+communication with the community for a specified period of time. No public or
+private interaction with the people involved, including unsolicited interaction
+with those enforcing the Code of Conduct, is allowed during this period.
+Violating these terms may lead to a permanent ban.
+
+### 4. Permanent Ban
+
+**Community Impact**: Demonstrating a pattern of violation of community
+standards, including sustained inappropriate behavior,  harassment of an
+individual, or aggression toward or disparagement of classes of individuals.
+
+**Consequence**: A permanent ban from any sort of public interaction within
+the community.
+
+## Attribution
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 2.0, available at
+[https://www.contributor-covenant.org/version/2/0/code_of_conduct.html][v2.0].
+
+Community Impact Guidelines were inspired by
+[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
+
+For answers to common questions about this code of conduct, see the FAQ at
+[https://www.contributor-covenant.org/faq][FAQ]. Translations are available
+at [https://www.contributor-covenant.org/translations][translations].
+
+[homepage]: https://www.contributor-covenant.org
+[v2.0]: https://www.contributor-covenant.org/version/2/0/code_of_conduct.html
+[Mozilla CoC]: https://github.com/mozilla/diversity
+[FAQ]: https://www.contributor-covenant.org/faq
+[translations]: https://www.contributor-covenant.org/translations
diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst
new file mode 100644
index 00000000..3484fefd
--- /dev/null
+++ b/CONTRIBUTING.rst
@@ -0,0 +1,91 @@
+.. _contribute:
+
+Contribute to EODAG development
+===============================
+
+Thank you for considering contributing to eodag!
+
+
+Reporting issues
+----------------
+
+Issue tracker: https://github.com/CS-SI/eodag/issues
+
+Please check that a similar issue does not already exist and
+include the following information in your post:
+
+-   Describe what you expected to happen.
+-   If possible, include a `minimal reproducible example`_ to help us
+    identify the issue. This also helps check that the issue is not with
+    your own code.
+-   Describe what actually happened. Include the full traceback if there
+    was an exception.
+-   List your Python and eodag versions. If possible, check if this
+    issue is already fixed in the latest releases or the latest code in
+    the repository.
+
+.. _minimal reproducible example: https://stackoverflow.com/help/minimal-reproducible-example
+
+
+Submitting patches
+------------------
+
+If you intend to contribute to eodag source code:
+
+.. code-block:: bash
+
+    git clone https://github.com/CS-SI/eodag.git
+    cd eodag
+    python -m pip install -r requirements-dev.txt
+    pre-commit install
+
+We use ``pre-commit`` to run a suite of linters, formatters and pre-commit hooks (``black``, ``isort``, ``flake8``) to ensure the code base is homogeneously formatted and easier to read. It's important that you install it, since we run the exact same hooks in the Continuous Integration.
+
+To run the default test suite (which excludes end-to-end tests):
+
+.. code-block:: bash
+
+    tox
+
+To only run end-to-end test:
+
+.. code-block:: bash
+
+    tox -- tests.test_end_to_end
+
+To run the entire tests (units, integraton and end-to-end):
+
+.. code-block:: bash
+
+    tox -- tests eodag
+
+.. note::
+
+    * Running the `tox` command will also build the docs. As The documentation
+      includes some notebooks (for the turorials), the build process will need
+      `pandoc <http://pandoc.org>`_ to succeed. If the build process fails for
+      you, please `install <http://pandoc.org/installing.html>`_ pandoc and try
+      again.
+
+    * When contributing to tutorials, you will need to keep notebook outputs
+      and save widget state. Otherwise outputs will not be visible in documentation.
+
+    * eodag is tested against python versions 3.6, 3.7, 3.8 and 3.9. Ensure you have
+      these versions installed before you run tox. You can use
+      `pyenv <https://github.com/pyenv/pyenv>`_ to manage many different versions
+      of python
+
+Releases are made by tagging a commit on the master branch. To make a new release,
+
+* Ensure you correctly updated `README.rst` and `CHANGES.rst` (and occasionally,
+  also `NOTICE` - in case a new dependency is added).
+* Check that the version string in `eodag/__meta__.py` (the variable `__version__`)
+  is correctly updated
+* Push your local master branch to remote.
+* Tag the commit that represents the state of the release with a message. For example,
+  for version 1.0, do this: `git tag -a v1.0 -m 'version 1.0'`
+* Push the tags to github: `git push --tags`.
+
+The documentation is managed by a webhook, and the latest documentation on readthedocs follows
+the documentation present in `develop`. Therefore, there is nothing to do apart from updating
+the `develop` branch to publish the latest documentation.
diff --git a/README.rst b/README.rst
index 686769ea..e5dde457 100644
--- a/README.rst
+++ b/README.rst
@@ -150,8 +150,17 @@ An eodag installation can be exposed through a STAC compliant REST api from the
     $ wget "http://127.0.0.1:5000/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items/S2A_MSIL1C_20210125T105331_N0209_R051_T31UCR_20210125T130733/download"
 
 
-You can also browse over your STAC API server using `STAC Browser <https://github.com/radiantearth/stac-browser>`_
-with ``CATALOG_URL=http://127.0.0.1:5000`` :
+You can also browse over your STAC API server using `STAC Browser <https://github.com/radiantearth/stac-browser>`_.
+Simply run:
+
+.. code-block:: bash
+
+    git clone https://github.com/CS-SI/eodag.git
+    cd eodag
+    docker-compose up
+
+
+And browse http://127.0.0.1:5001:
 
 .. image:: https://raw.githubusercontent.com/CS-SI/eodag/develop/docs/_static/stac_browser_example_600.png
    :target: https://raw.githubusercontent.com/CS-SI/eodag/develop/docs/_static/stac_browser_example.png
@@ -178,22 +187,23 @@ Then you can start playing with it:
         --start 2018-01-01 \
         --end 2018-01-31 \
         --cloudCover 20 \
-        --productType S2_MSI_L1C
-        --cruncher FilterLatestIntersect \
+        --productType S2_MSI_L1C \
+        --all \
         --storage my_search.geojson
 
-The request above search for product types `S2_MSI_L1C` and will crunch the result using cruncher `FilterLatestIntersect`
-and storing the overall result to `my_search.geojson`.
+The request above searches for `S2_MSI_L1C` product types in a given bounding box, in January 2018. The command fetches internally all
+the products that match these criteria. Without `--all`, it would only fetch the products found on the first result page.
+It finally saves the results in a GeoJSON file.
 
 You can pass arguments to a cruncher on the command line by doing this (example with using `FilterOverlap` cruncher
 which takes `minimum_overlap` as argument)::
 
-        eodag search -f my_conf.yml -b 1 43 2 44 -s 2018-01-01 -e 2018-01-31 -p S2_MSI_L1C \
+        eodag search -f my_conf.yml -b 1 43 2 44 -s 2018-01-01 -e 2018-01-31 -p S2_MSI_L1C --all \
                      --cruncher FilterOverlap \
                      --cruncher-args FilterOverlap minimum_overlap 10
 
 The request above means : "Give me all the products of type `S2_MSI_L1C`, use `FilterOverlap` to keep only those products
-that are contained in the bbox I gave you, or whom spatial extent overlaps at least 10% (`minimum_overlap`) of the surface
+that are contained in the bbox I gave you, or whose spatial extent overlaps at least 10% (`minimum_overlap`) of the surface
 of this bbox"
 
 * To download the result of a previous call to `search`::
@@ -219,56 +229,11 @@ of this bbox"
 Contribute
 ==========
 
-If you intend to contribute to eodag source code::
-
-    git clone https://github.com/CS-SI/eodag.git
-    cd eodag
-    python -m pip install -r requirements-dev.txt
-    pre-commit install
-
-To run the default test suite (which excludes end-to-end tests)::
-
-    tox
-
-To only run end-to-end test::
-
-    tox -- tests.test_end_to_end
-
-To run the entire tests (units, integration and end-to-end)::
-
-    tox -- tests eodag
-
-
-.. note::
-
-    * Running the `tox` command will also build the docs. As The documentation
-      includes some notebooks (for the turorials), the build process will need
-      `pandoc <http://pandoc.org>`_ to succeed. If the build process fails for
-      you, please `install <http://pandoc.org/installing.html>`_ pandoc and try
-      again.
-
-    * When contributing to tutorials, you will need to keep notebook outputs
-      and save widget state. Otherwise outputs will not be visible in documentation.
-
-    * eodag is tested against python versions 3.6, 3.7, 3.8 and 3.9. Ensure you have
-      these versions installed before you run tox. You can use
-      `pyenv <https://github.com/pyenv/pyenv>`_ to manage many different versions
-      of python
-
-Releases are made by tagging a commit on the master branch. To make a new release,
+For guidance on setting up a development environment and how to make a
+contribution to eodag, see the `contributing guidelines`_.
 
-* Ensure you correctly updated `README.rst` and `CHANGES.rst` (and occasionally,
-  also `NOTICE` - in case a new dependency is added).
-* Check that the version string in `eodag/__meta__.py` (the variable `__version__`)
-  is correctly updated
-* Push your local master branch to remote.
-* Tag the commit that represents the state of the release with a message. For example,
-  for version 1.0, do this: `git tag -a v1.0 -m 'version 1.0'`
-* Push the tags to github: `git push --tags`.
+.. _contributing guidelines: https://github.com/CS-SI/eodag/blob/develop/CONTRIBUTING.rst
 
-The documentation is managed by a webhook, and the latest documentation on readthedocs follows
-the documentation present in `master`. Therefore, there is nothing to do apart from updating
-the `master` branch to publish the latest documentation.
 
 LICENSE
 =======
diff --git a/docker-compose.yml b/docker-compose.yml
new file mode 100644
index 00000000..504d3f4e
--- /dev/null
+++ b/docker-compose.yml
@@ -0,0 +1,27 @@
+version: "3.7"
+
+networks:
+  eodag-stac-network:
+    driver: bridge
+
+services:
+  browser:
+    build:
+      context: .
+      dockerfile: docker/stac-browser.dockerfile
+    container_name: stac_browser
+    restart: unless-stopped
+    networks:
+      - eodag-stac-network
+    ports:
+      - "5001:80"
+  stac:
+    build:
+      context: .
+      dockerfile: docker/stac-server.dockerfile
+    container_name: stac_server
+    restart: unless-stopped
+    networks:
+      - eodag-stac-network
+    ports:
+      - "5000:5000"
diff --git a/docker/run-stac-server.sh b/docker/run-stac-server.sh
new file mode 100644
index 00000000..c1754e35
--- /dev/null
+++ b/docker/run-stac-server.sh
@@ -0,0 +1,6 @@
+#! /bin/bash
+
+# some pre-start operation
+
+# start
+eodag serve-rest -w
diff --git a/docker/stac-browser.dockerfile b/docker/stac-browser.dockerfile
new file mode 100644
index 00000000..95c27012
--- /dev/null
+++ b/docker/stac-browser.dockerfile
@@ -0,0 +1,27 @@
+# build stage
+FROM node:lts-jessie-slim as build-stage
+
+# install git
+RUN apt-get update \
+    && DEBIAN_FRONTEND=noninteractive apt-get install -y git
+
+#clone repository
+RUN git clone https://github.com/radiantearth/stac-browser.git
+
+# move to folder
+WORKDIR /stac-browser
+
+# install
+RUN npm install
+
+# start application
+RUN CATALOG_URL=http://localhost:5000 npm run build
+
+# production stage, self describing
+FROM nginx:stable-alpine as production-stage
+
+COPY --from=build-stage /stac-browser/dist /usr/share/nginx/html
+
+EXPOSE 80
+
+CMD ["nginx", "-g", "daemon off;"]
diff --git a/docker/stac-server.dockerfile b/docker/stac-server.dockerfile
new file mode 100644
index 00000000..db4c9f33
--- /dev/null
+++ b/docker/stac-server.dockerfile
@@ -0,0 +1,74 @@
+FROM python:3.9-slim
+
+# start from root
+WORKDIR /
+
+ENV \
+    # force stdin, stdout and stderr to be totally unbuffered. (equivalent to `python -u`)
+    PYTHONUNBUFFERED=1 \
+    # prevents python creating .pyc files(equivalent to `python -B`)
+    PYTHONDONTWRITEBYTECODE=1 \
+    # enable hash randomization (equivalent to `python -R`)
+    PYTHONHASHSEED=random \
+    # fault handler (equivalent to `python -X`)
+    PYTHONFAULTHANDLER=1 \
+    # python encoding
+    PYTHONIOENCODING=UTF-8 \
+    \
+    # set pip settings
+    PIP_NO_CACHE_DIR=off \
+    PIP_DISABLE_PIP_VERSION_CHECK=on \
+    PIP_DEFAULT_TIMEOUT=100 \
+    # set time.zone
+    TZ=UTC
+
+# update system
+RUN apt-get update \
+    && apt-get upgrade -y
+
+# reconfigure timezone
+RUN echo $TZ > /etc/timezone && \
+    apt-get install -y tzdata && \
+    rm /etc/localtime && \
+    ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && \
+    dpkg-reconfigure -f noninteractive tzdata && \
+    apt-get clean
+
+# install locales
+RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y locales
+
+# ensure locales are configured correctly
+RUN sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && \
+    dpkg-reconfigure --frontend=noninteractive locales && \
+    update-locale LANG=en_US.UTF-8
+
+ENV LC_ALL=en_US.UTF-8 \
+    LANG=en_US.UTF-8
+
+# copy necessary files
+COPY setup.py setup.py
+COPY setup.cfg setup.cfg
+COPY README.rst README.rst
+COPY ./eodag /eodag
+
+# install eodag
+RUN python setup.py install
+
+# add python path
+ENV PYTHONPATH="${PYTHONPATH}:/eodag"
+
+# copy start-stac script
+COPY ./docker/run-stac-server.sh /eodag/run-stac-server.sh
+
+# and make executable
+RUN chmod +x /eodag/run-stac-server.sh
+
+# add user
+RUN addgroup --system user \
+    && adduser --system --group user
+
+# switch to non-root user
+USER user
+
+# and then start STAC
+CMD ["/eodag/run-stac-server.sh"]
diff --git a/docs/_static/custom.css b/docs/_static/custom.css
new file mode 100644
index 00000000..6a54a42e
--- /dev/null
+++ b/docs/_static/custom.css
@@ -0,0 +1,4 @@
+div.sphinxsidebar {
+    max-height: 100%;
+    overflow-y: auto;
+}
diff --git a/docs/conf.py b/docs/conf.py
index 5357ea66..3c6fddcb 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -111,7 +111,7 @@ html_theme_options = {
     "logo": "eodag_logo.png",
     "logo_name": False,
     "show_related": True,
-    "fixed_sidebar": False,
+    "fixed_sidebar": True,
     "show_powered_by": False,
     "github_user": "CS-SI",
     "github_repo": "eodag",
diff --git a/docs/contribute.rst b/docs/contribute.rst
index 654e93f8..e582053e 100644
--- a/docs/contribute.rst
+++ b/docs/contribute.rst
@@ -1,61 +1,1 @@
-.. _contribute:
-
-Contribute to EODAG development
-===============================
-
-If you intend to contribute to eodag source code:
-
-.. code-block:: bash
-
-    git clone https://github.com/CS-SI/eodag.git
-    cd eodag
-    python -m pip install -r requirements-dev.txt
-
-To run the default test suite (which excludes end-to-end tests):
-
-.. code-block:: bash
-
-    tox
-
-To only run end-to-end test:
-
-.. code-block:: bash
-
-    tox -- tests.test_end_to_end
-
-To run the entire tests (units, integraton and end-to-end):
-
-.. code-block:: bash
-
-    tox -- tests eodag
-
-.. note::
-
-    * Running the `tox` command will also build the docs. As The documentation
-      includes some notebooks (for the turorials), the build process will need
-      `pandoc <http://pandoc.org>`_ to succeed. If the build process fails for
-      you, please `install <http://pandoc.org/installing.html>`_ pandoc and try
-      again.
-
-    * When contributing to tutorials, you will need to keep notebook outputs
-      and save widget state. Otherwise outputs will not be visible in documentation.
-
-    * eodag is tested against python versions 3.6, 3.7, 3.8 and 3.9. Ensure you have
-      these versions installed before you run tox. You can use
-      `pyenv <https://github.com/pyenv/pyenv>`_ to manage many different versions
-      of python
-
-Releases are made by tagging a commit on the master branch. To make a new release,
-
-    * Ensure you correctly updated `README.rst` and `CHANGES.rst` (and occasionally,
-      also `NOTICE` - in case a new dependency is added).
-    * Check that the version string in `eodag/__meta__.py` (the variable `__version__`)
-      is correctly updated
-    * Push your local master branch to remote.
-    * Tag the commit that represents the state of the release with a message. For example,
-      for version 1.0, do this: `git tag -a v1.0 -m 'version 1.0'`
-    * Push the tags to github: `git push --tags`.
-
-The documentation is managed by a webhook, and the latest documentation on readthedocs follows
-the documentation present in `master`. Therefore, there is nothing to do apart from updating
-the `master` branch to publish the latest documentation.
+.. include:: ../CONTRIBUTING.rst
diff --git a/docs/intro.rst b/docs/intro.rst
index d45e5001..4ad91e68 100644
--- a/docs/intro.rst
+++ b/docs/intro.rst
@@ -45,13 +45,11 @@ STAC client and server
 EODAG's interfaces are compliant with the `SpatioTemporal Asset Catalog <https://github.com/radiantearth/stac-spec>`_
 (STAC) specification:
 
-    * `STAC client <tutorials/tuto_stac_client.nblink>`_: STAC API providers can be configured to be used for `search` and `download` using EODAG. One
-      provider (astraea_eod) is already implemented. New providers can be dynamically added by the user. Static
-      catalogs can also be fetched by EODAG. *The search plugin for static catalogs is under development, but some
-      methods are already available to enable working with these catalogs.* See
+    * `STAC client <tutorials/tuto_stac_client.nblink>`_: STAC API providers can be configured to be used for `search`
+      and `download` using EODAG. Some providers (astraea_eod, earth_search, usgs_satapi_aws) are already implemented
+      and new providers can be dynamically added by the user. Static catalogs can also be fetched by EODAG. See
       `STAC client tutorial <tutorials/tuto_stac_client.nblink>`_.
 
-
     * `STAC server <use.html#stac-rest-interface>`_: EODAG can run as STAC API REST server and give access to configured
       providers data through a STAC compliant search API. See `usage section <use.html#stac-rest-interface>`_
       for more details.
diff --git a/docs/use.rst b/docs/use.rst
index ef129fd0..db417cf0 100644
--- a/docs/use.rst
+++ b/docs/use.rst
@@ -91,7 +91,16 @@ Example URLs:
   products over France, aquired during October 2019, and having 10% maximum cloud cover
 
 Browsing over catalogs can be experienced connecting EODAG STAC API to
-`STAC-Browser <https://github.com/radiantearth/stac-browser>`_:
+`STAC-Browser <https://github.com/radiantearth/stac-browser>`_. Simply run:
+
+.. code-block:: bash
+
+    git clone https://github.com/CS-SI/eodag.git
+    cd eodag
+    docker-compose up
+
+
+And browse http://127.0.0.1:5001:
 
 .. image:: _static/stac_browser_example.png
    :width: 800
@@ -172,23 +181,24 @@ Then you can start playing with it:
         --box 1 43 2 44 \
         --start 2018-01-01 --end 2018-01-31 \
         --productType S2_MSI_L1C \
-        --cruncher FilterLatestIntersect \
+        --all \
         --storage my_search.geojson
 
-The request above search for product types `S2_MSI_L1C` and will crunch the result using cruncher `FilterLatestIntersect`
-and storing the overall result to `my_search.geojson`.
+The request above searches for `S2_MSI_L1C` product types in a given bounding box, in January 2018. The command fetches internally all
+the products that match these criteria. Without `--all`, it would only fetch the products found on the first result page.
+It finally saves the results in a GeoJSON file.
 
 You can pass arguments to a cruncher on the command line by doing this (example with using `FilterOverlap` cruncher
 which takes `minimum_overlap` as argument):
 
 .. code-block:: console
 
-        eodag search -f my_conf.yml -b 1 43 2 44 -s 2018-01-01 -e 2018-01-31 -p S2_MSI_L1C \
+        eodag search -f my_conf.yml -b 1 43 2 44 -s 2018-01-01 -e 2018-01-31 -p S2_MSI_L1C --all \
                      --cruncher FilterOverlap \
                      --cruncher-args FilterOverlap minimum_overlap 10
 
 The request above means : "Give me all the products of type `S2_MSI_L1C`, use `FilterOverlap` to keep only those products
-that are contained in the bbox I gave you, or whom spatial extent overlaps at least 10% (`minimum_overlap`) of the surface
+that are contained in the bbox I gave you, or whose spatial extent overlaps at least 10% (`minimum_overlap`) of the surface
 of this bbox"
 
 You can use `eaodag search` with custom parameters. Custom parameters will be used as is in the query string search sent
diff --git a/eodag/__meta__.py b/eodag/__meta__.py
index 9e090d98..6a9288c9 100644
--- a/eodag/__meta__.py
+++ b/eodag/__meta__.py
@@ -17,7 +17,7 @@
 # limitations under the License.
 __title__ = "eodag"
 __description__ = "Earth Observation Data Access Gateway"
-__version__ = "2.1.1"
+__version__ = "2.2.0"
 __author__ = "CS GROUP - France (CSSI)"
 __author_email__ = "admin@geostorm.eu"
 __url__ = "https://github.com/CS-SI/eodag"
diff --git a/eodag/api/core.py b/eodag/api/core.py
index 8c8ebe98..31a214cd 100644
--- a/eodag/api/core.py
+++ b/eodag/api/core.py
@@ -59,6 +59,10 @@ logger = logging.getLogger("eodag.core")
 # pagination defaults
 DEFAULT_PAGE = 1
 DEFAULT_ITEMS_PER_PAGE = 20
+# Default maximum number of items per page requested by search_all. 50 instead of
+# 20 (DEFAULT_ITEMS_PER_PAGE) to increase it to the known and currentminimum
+# value (mundi)
+DEFAULT_MAX_ITEMS_PER_PAGE = 50
 
 
 class EODataAccessGateway(object):
@@ -100,6 +104,8 @@ class EODataAccessGateway(object):
         override_config_from_env(self.providers_config)
 
         self._plugins_manager = PluginManager(self.providers_config)
+        # use updated providers_config
+        self.providers_config = self._plugins_manager.providers_config
 
         # Build a search index for product types
         self._product_types_index = None
@@ -119,9 +125,13 @@ class EODataAccessGateway(object):
                     with open(locations_conf_template) as infile, open(
                         locations_conf_path, "w"
                     ) as outfile:
+                        # The template contains paths in the form of:
+                        # /path/to/locations/file.shp
+                        path_template = "/path/to/locations/"
                         for line in infile:
                             line = line.replace(
-                                "/path/to/locations", os.path.join(self.conf_dir, "shp")
+                                path_template,
+                                os.path.join(self.conf_dir, "shp") + os.path.sep,
                             )
                             outfile.write(line)
                     # copy sample shapefile dir
@@ -205,7 +215,7 @@ class EODataAccessGateway(object):
         """Set max priority for the given provider.
 
         >>> import tempfile, os
-        >>> config = tempfile.NamedTemporaryFile(delete=True)
+        >>> config = tempfile.NamedTemporaryFile(delete=False)
         >>> dag = EODataAccessGateway(user_conf_file_path=os.path.join(
         ...     tempfile.gettempdir(), config.name))
         >>> # This also tests get_preferred_provider method by the way
@@ -230,13 +240,16 @@ class EODataAccessGateway(object):
         >>> dag.get_preferred_provider()
         ('usgs', 4)
         >>> config.close()
+        >>> os.unlink(config.name)
 
         :param provider: The name of the provider that should be considered as the
                          preferred provider to be used for this instance
         :type provider: str
         """
         if provider not in self.available_providers():
-            raise UnsupportedProvider("This provider is not recognised by eodag")
+            raise UnsupportedProvider(
+                f"This provider is not recognised by eodag: {provider}"
+            )
         preferred_provider, max_priority = self.get_preferred_provider()
         if preferred_provider != provider:
             new_priority = max_priority + 1
@@ -384,22 +397,6 @@ class EODataAccessGateway(object):
     def guess_product_type(self, **kwargs):
         """Find the eodag product type code that best matches a set of search params
 
-        >>> from eodag import EODataAccessGateway
-        >>> dag = EODataAccessGateway()
-        >>> dag.guess_product_type(
-        ...     instrument="MSI",
-        ...     platform="SENTINEL2",
-        ...     platformSerialIdentifier="S2A",
-        ... ) # doctest: +NORMALIZE_WHITESPACE
-        ['S2_MSI_L1C', 'S2_MSI_L2A', 'S2_MSI_L2A_MAJA', 'S2_MSI_L2B_MAJA_SNOW',
-            'S2_MSI_L2B_MAJA_WATER', 'S2_MSI_L3A_WASP']
-        >>> import eodag.utils.exceptions
-        >>> try:
-        ...     dag.guess_product_type()
-        ...     raise AssertionError(u"NoMatchingProductType exception not raised")
-        ... except eodag.utils.exceptions.NoMatchingProductType:
-        ...     pass
-
         :param kwargs: A set of search parameters as keywords arguments
         :return: The best match for the given parameters
         :rtype: str
@@ -431,7 +428,7 @@ class EODataAccessGateway(object):
                     kwargs[search_key]
                 )
                 if results is None:
-                    results = searcher.search(query)
+                    results = searcher.search(query, limit=None)
                 else:
                     results.upgrade_and_extend(searcher.search(query))
             guesses = [r["ID"] for r in results or []]
@@ -448,7 +445,7 @@ class EODataAccessGateway(object):
         end=None,
         geom=None,
         locations=None,
-        **kwargs
+        **kwargs,
     ):
         """Look for products matching criteria on known providers.
 
@@ -550,6 +547,295 @@ class EODataAccessGateway(object):
             return a list as a result of their processing. This requirement is
             enforced here.
         """
+        search_kwargs = self._prepare_search(
+            start=start, end=end, geom=geom, locations=locations, **kwargs
+        )
+        if search_kwargs.get("id"):
+            provider = search_kwargs.get("provider")
+            return self._search_by_id(search_kwargs["id"], provider)
+        search_plugin = search_kwargs.pop("search_plugin")
+        search_kwargs.update(
+            page=page,
+            items_per_page=items_per_page,
+        )
+        return self._do_search(
+            search_plugin, count=True, raise_errors=raise_errors, **search_kwargs
+        )
+
+    def search_iter_page(
+        self,
+        items_per_page=DEFAULT_ITEMS_PER_PAGE,
+        start=None,
+        end=None,
+        geom=None,
+        locations=None,
+        **kwargs,
+    ):
+        """Iterate over the pages of a products search.
+
+        :param items_per_page: The number of results requested per page (default: 20)
+        :type items_per_page: int
+        :param start: Start sensing UTC time in iso format
+        :type start: str
+        :param end: End sensing UTC time in iso format
+        :type end: str
+        :param geom: Search area that can be defined in different ways:
+
+                    * with a Shapely geometry object:
+                      ``class:`shapely.geometry.base.BaseGeometry```
+                    * with a bounding box (dict with keys: "lonmin", "latmin", "lonmax", "latmax"):
+                      ``dict.fromkeys(["lonmin", "latmin", "lonmax", "latmax"])``
+                    * with a bounding box as list of float:
+                      ``[lonmin, latmin, lonmax, latmax]``
+                    * with a WKT str
+
+        :type geom: Union[str, dict, shapely.geometry.base.BaseGeometry]
+        :param locations: Location filtering by name using locations configuration
+                          ``{"<location_name>"="<attr_regex>"}``. For example, ``{"country"="PA."}`` will use
+                          the geometry of the features having the property ISO3 starting with
+                          'PA' such as Panama and Pakistan in the shapefile configured with
+                          name=country and attr=ISO3
+        :type locations: dict
+        :param dict kwargs: some other criteria that will be used to do the search,
+                            using paramaters compatibles with the provider
+        :returns: An iterator that yields page per page a collection of EO products
+                  matching the criteria
+        :rtype: Iterator[:class:`~eodag.api.search_result.SearchResult`]
+
+        .. versionadded::
+            2.2.0
+        """
+        search_kwargs = self._prepare_search(
+            start=start, end=end, geom=geom, locations=locations, **kwargs
+        )
+        search_plugin = search_kwargs.pop("search_plugin")
+        iteration = 1
+        # Store the search plugin config pagination.next_page_url_tpl to reset it later
+        # since it might be modified if the next_page_url mechanism is used by the
+        # plugin.
+        pagination_config = getattr(search_plugin.config, "pagination", {})
+        prev_next_page_url_tpl = pagination_config.get("next_page_url_tpl")
+        # Page has to be set to a value even if use_next is True, this is required
+        # internally by the search plugin (see collect_search_urls)
+        search_kwargs.update(
+            page=1,
+            items_per_page=items_per_page,
+        )
+        prev_product = None
+        next_page_url = None
+        while True:
+            if iteration > 1 and next_page_url:
+                pagination_config["next_page_url_tpl"] = next_page_url
+            logger.debug("Iterate over pages: search page %s", iteration)
+            try:
+                products, _ = self._do_search(
+                    search_plugin, count=False, raise_errors=True, **search_kwargs
+                )
+            finally:
+                # we don't want that next(search_iter_page(...)) modifies the plugin
+                # indefinitely. So we reset after each request, but before the generator
+                # yields, the attr next_page_url (to None) and
+                # config.pagination["next_page_url_tpl"] (to its original value).
+                next_page_url = getattr(search_plugin, "next_page_url", None)
+                if next_page_url:
+                    search_plugin.next_page_url = None
+                    if prev_next_page_url_tpl:
+                        search_plugin.config.pagination[
+                            "next_page_url_tpl"
+                        ] = prev_next_page_url_tpl
+            if len(products) > 0:
+                # The first products between two iterations are compared. If they
+                # are actually the same product, it means the iteration failed at
+                # progressing for some reason. This is implemented as a workaround
+                # to some search plugins/providers not handling pagination.
+                product = products[0]
+                if (
+                    prev_product
+                    and product.properties["id"] == prev_product.properties["id"]
+                    and product.provider == prev_product.provider
+                ):
+                    logger.warning(
+                        "Iterate over pages: stop iterating since the next page "
+                        "appears to have the same products as in the previous one. "
+                        "This provider may not implement pagination.",
+                    )
+                    last_page_with_products = iteration - 1
+                    break
+                yield products
+                prev_product = product
+                # Prevent a last search if the current one returned less than the
+                # maximum number of items asked for.
+                if len(products) < items_per_page:
+                    last_page_with_products = iteration
+                    break
+            else:
+                last_page_with_products = iteration - 1
+                break
+            iteration += 1
+            search_kwargs["page"] = iteration
+        logger.debug(
+            "Iterate over pages: last products found on page %s",
+            last_page_with_products,
+        )
+
+    def search_all(
+        self,
+        items_per_page=None,
+        start=None,
+        end=None,
+        geom=None,
+        locations=None,
+        **kwargs,
+    ):
+        """Search and return all the products matching the search criteria.
+
+        It iterates over the pages of a search query and collects all the returned
+        products into a single :class:`~eodag.api.search_result.SearchResult` instance.
+
+        :param items_per_page: (optional) The number of results requested internally per
+                               page. The maximum number of items than can be requested
+                               at once to a provider has been configured in EODAG for
+                               some of them. If items_per_page is None and this number
+                               is available for the searched provider, it is used to
+                               limit the number of requests made. This should also
+                               reduce the time required to collect all the products
+                               matching the search criteria. If this number is not
+                               available, a default value of 50 is used instead.
+                               items_per_page can also be set to any arbitrary value.
+        :type items_per_page: int
+        :param start: Start sensing UTC time in iso format
+        :type start: str
+        :param end: End sensing UTC time in iso format
+        :type end: str
+        :param geom: Search area that can be defined in different ways:
+
+                    * with a Shapely geometry object:
+                      ``class:`shapely.geometry.base.BaseGeometry```
+                    * with a bounding box (dict with keys: "lonmin", "latmin", "lonmax", "latmax"):
+                      ``dict.fromkeys(["lonmin", "latmin", "lonmax", "latmax"])``
+                    * with a bounding box as list of float:
+                      ``[lonmin, latmin, lonmax, latmax]``
+                    * with a WKT str
+
+        :type geom: Union[str, dict, shapely.geometry.base.BaseGeometry]
+        :param locations: Location filtering by name using locations configuration
+                          ``{"<location_name>"="<attr_regex>"}``. For example, ``{"country"="PA."}`` will use
+                          the geometry of the features having the property ISO3 starting with
+                          'PA' such as Panama and Pakistan in the shapefile configured with
+                          name=country and attr=ISO3
+        :type locations: dict
+        :param dict kwargs: some other criteria that will be used to do the search,
+                            using paramaters compatibles with the provider
+        :returns: An iterator that yields page per page a collection of EO products
+                  matching the criteria
+        :rtype: Iterator[:class:`~eodag.api.search_result.SearchResult`]
+
+        .. versionadded::
+            2.2.0
+        """
+        # Prepare the search just to get the search plugin and the maximized value
+        # of items_per_page if defined for the provider used.
+        search_kwargs = self._prepare_search(
+            start=start, end=end, geom=geom, locations=locations, **kwargs
+        )
+        search_plugin = search_kwargs["search_plugin"]
+        if items_per_page is None:
+            items_per_page = search_plugin.config.pagination.get(
+                "max_items_per_page", DEFAULT_MAX_ITEMS_PER_PAGE
+            )
+        logger.debug(
+            "Searching for all the products with provider %s and a maximum of %s "
+            "items per page.",
+            search_plugin.provider,
+            items_per_page,
+        )
+        all_results = SearchResult([])
+        for page_results in self.search_iter_page(
+            items_per_page=items_per_page,
+            start=start,
+            end=end,
+            geom=geom,
+            locations=locations,
+            **kwargs,
+        ):
+            all_results.data.extend(page_results.data)
+        logger.info(
+            "Found %s result(s) on provider '%s'",
+            len(all_results),
+            search_plugin.provider,
+        )
+        return all_results
+
+    def _search_by_id(self, uid, provider=None):
+        """Internal method that enables searching a product by its id.
+
+        Keeps requesting providers until a result matching the id is supplied. The
+        search plugins should be developed in the way that enable them to handle the
+        support of a search by id by the providers. The providers are requested one by
+        one, in the order defined by their priorities. Be aware that because of that,
+        the search can be slow, if the priority order is such that the provider that
+        contains the requested product has the lowest priority. However, you can always
+        speed up a little the search by passing the name of the provider on which to
+        perform the search, if this information is available
+
+        :param uid: The uid of the EO product
+        :type uid: str
+        :param provider: (optional) The provider on which to search the product.
+                         This may be useful for performance reasons when the user
+                         knows this product is available on the given provider
+        :type provider: str
+        :returns: A search result with one EO product or None at all, and the number
+                  of EO products retrieved (0 or 1)
+        :rtype: tuple(:class:`~eodag.api.search_result.SearchResult`, int)
+
+        .. versionadded:: 1.0
+        """
+        for plugin in self._plugins_manager.get_search_plugins(provider=provider):
+            logger.info(
+                "Searching product with id '%s' on provider: %s", uid, plugin.provider
+            )
+            logger.debug("Using plugin class for search: %s", plugin.__class__.__name__)
+            auth = self._plugins_manager.get_auth_plugin(plugin.provider)
+            results, _ = self._do_search(plugin, auth=auth, id=uid)
+            if len(results) == 1:
+                return results, 1
+        return SearchResult([]), 0
+
+    def _prepare_search(
+        self, start=None, end=None, geom=None, locations=None, **kwargs
+    ):
+        """Internal method to prepare the search kwargs and get the search
+        and auth plugins.
+
+        Product query:
+          * By id (plus optional 'provider')
+          * By search params:
+            * productType query:
+              * By product type (e.g. 'S2_MSI_L1C')
+              * By params (e.g. 'platform'), see guess_product_type
+            * dates: 'start' and/or 'end'
+            * geometry: 'geom' or 'bbox' or 'box'
+            * search locations
+            * TODO: better expose cloudCover
+            * other search params are passed to Searchplugin.query()
+
+        :param start: Start sensing UTC time in iso format
+        :type start: str
+        :param end: End sensing UTC time in iso format
+        :type end: str
+        :param geom: Search area that can be defined in different ways (see search)
+        :type geom: Union[str, dict, shapely.geometry.base.BaseGeometry]
+        :param locations: Location filtering by name using locations configuration
+        :type locations: dict
+        :param dict kwargs: Some other criteria
+                            * id and/or a provider for a search by
+                            * search criteria to guess the product type
+                            * other criteria compatible with the provider
+        :returns: The prepared kwargs to make a query.
+        :rtype: dict
+
+        .. versionadded:: 2.2
+        """
         product_type = kwargs.get("productType", None)
         if product_type is None:
             try:
@@ -563,14 +849,18 @@ class EODataAccessGateway(object):
                         "No product type could be guessed with provided arguments"
                     )
                 else:
-                    provider = kwargs.get("provider", None)
-                    return self._search_by_id(kwargs["id"], provider=provider)
+                    return kwargs
 
         kwargs["productType"] = product_type
         if start is not None:
             kwargs["startTimeFromAscendingNode"] = start
         if end is not None:
             kwargs["completionTimeFromAscendingNode"] = end
+        if "box" in kwargs or "bbox" in kwargs:
+            logger.warning(
+                "'box' or 'bbox' parameters are only supported for backwards "
+                " compatibility reasons. Usage of 'geom' is recommended."
+            )
         if geom is not None:
             kwargs["geometry"] = geom
         box = kwargs.pop("box", None)
@@ -584,96 +874,75 @@ class EODataAccessGateway(object):
         locations_dict = {loc["name"]: loc for loc in self.locations_config}
         for arg in locations_dict.keys():
             kwargs.pop(arg, None)
+        del kwargs["locations"]
 
-        plugin = next(
+        search_plugin = next(
             self._plugins_manager.get_search_plugins(product_type=product_type)
         )
         logger.info(
-            "Searching product type '%s' on provider: %s", product_type, plugin.provider
+            "Searching product type '%s' on provider: %s",
+            product_type,
+            search_plugin.provider,
         )
-        # add product_types_config to plugin config
+        # Add product_types_config to plugin config. This dict contains product
+        # type metadata that will also be stored in each product's properties.
         try:
-            plugin.config.product_type_config = dict(
+            search_plugin.config.product_type_config = dict(
                 [
                     p
-                    for p in self.list_product_types(plugin.provider)
+                    for p in self.list_product_types(search_plugin.provider)
                     if p["ID"] == product_type
                 ][0],
-                **{"productType": product_type}
+                **{"productType": product_type},
             )
+        # If the product isn't in the catalog, it's a generic product type.
         except IndexError:
-            plugin.config.product_type_config = dict(
+            search_plugin.config.product_type_config = dict(
                 [
                     p
-                    for p in self.list_product_types(plugin.provider)
+                    for p in self.list_product_types(search_plugin.provider)
                     if p["ID"] == GENERIC_PRODUCT_TYPE
                 ][0],
-                **{"productType": product_type}
+                **{"productType": product_type},
             )
-        plugin.config.product_type_config.pop("ID", None)
+        # Remove the ID since this is equal to productType.
+        search_plugin.config.product_type_config.pop("ID", None)
 
-        logger.debug("Using plugin class for search: %s", plugin.__class__.__name__)
-        auth = self._plugins_manager.get_auth_plugin(plugin.provider)
-        return self._do_search(
-            plugin,
-            auth=auth,
-            page=page,
-            items_per_page=items_per_page,
-            raise_errors=raise_errors,
-            **kwargs
+        logger.debug(
+            "Using plugin class for search: %s", search_plugin.__class__.__name__
         )
+        auth_plugin = self._plugins_manager.get_auth_plugin(search_plugin.provider)
 
-    def _search_by_id(self, uid, provider=None):
-        """Internal method that enables searching a product by its id.
-
-        Keeps requesting providers until a result matching the id is supplied. The
-        search plugins should be developed in the way that enable them to handle the
-        support of a search by id by the providers. The providers are requested one by
-        one, in the order defined by their priorities. Be aware that because of that,
-        the search can be slow, if the priority order is such that the provider that
-        contains the requested product has the lowest priority. However, you can always
-        speed up a little the search by passing the name of the provider on which to
-        perform the search, if this information is available
+        return dict(search_plugin=search_plugin, auth=auth_plugin, **kwargs)
 
-        :param uid: The uid of the EO product
-        :type uid: str
-        :param provider: (optional) The provider on which to search the product.
-                         This may be useful for performance reasons when the user
-                         knows this product is available on the given provider
-        :type provider: str
-        :returns: A search result with one EO product or None at all, and the number
-                  of EO products retrieved (0 or 1)
-        :rtype: tuple(:class:`~eodag.api.search_result.SearchResult`, int)
-
-        .. versionadded:: 1.0
-        """
-        for plugin in self._plugins_manager.get_search_plugins(provider=provider):
-            logger.info(
-                "Searching product with id '%s' on provider: %s", uid, plugin.provider
-            )
-            logger.debug("Using plugin class for search: %s", plugin.__class__.__name__)
-            auth = self._plugins_manager.get_auth_plugin(plugin.provider)
-            results, _ = self._do_search(plugin, auth=auth, id=uid)
-            if len(results) == 1:
-                return results, 1
-        return SearchResult([]), 0
-
-    def _do_search(self, search_plugin, **kwargs):
+    def _do_search(self, search_plugin, count=True, raise_errors=False, **kwargs):
         """Internal method that performs a search on a given provider.
 
+        :param search_plugin: A search plugin
+        :type search_plugin: eodag.plugins.base.Search
+        :param count: Whether to run a query with a count request or not (default: True)
+        :type count: bool
+        :param raise_errors:  When an error occurs when searching, if this is set to
+                              True, the error is raised (default: False)
+        :type raise_errors: bool
+        :param dict kwargs: some other criteria that will be used to do the search
+        :returns: A collection of EO products matching the criteria and the total
+                  number of results found if count is True else None
+        :rtype: tuple(:class:`~eodag.api.search_result.SearchResult`, int or None)
+
         .. versionadded:: 1.0
         """
         results = SearchResult([])
         total_results = 0
         try:
-            res, nb_res = search_plugin.query(**kwargs)
+            res, nb_res = search_plugin.query(count=count, **kwargs)
 
             # Only do the pagination computations when it makes sense. For example,
             # for a search by id, we can reasonably guess that the provider will return
             # At most 1 product, so we don't need such a thing as pagination
             page = kwargs.get("page")
             items_per_page = kwargs.get("items_per_page")
-            if page and items_per_page:
+            if page and items_per_page and count:
                 # Take into account the fact that a provider may not return the count of
                 # products (in that case, fallback to using the length of the results it
                 # returned and the page requested. As an example, check the result of
@@ -731,31 +1000,34 @@ class EODataAccessGateway(object):
                     )
 
             results.extend(res)
-            total_results += nb_res
-            logger.info(
-                "Found %s result(s) on provider '%s'", nb_res, search_plugin.provider
-            )
-            # Hitting for instance
-            # https://theia.cnes.fr/atdistrib/resto2/api/collections/SENTINEL2/
-            #   search.json?startDate=2019-03-01&completionDate=2019-06-15
-            #   &processingLevel=LEVEL2A&maxRecords=1&page=1
-            # returns a number (properties.totalResults) that is the number of
-            # products in the collection (here SENTINEL2) instead of the estimated
-            # total number of products matching the search criteria (start/end date).
-            # Remove this warning when this is fixed upstream by THEIA.
-            if search_plugin.provider == "theia":
-                logger.warning(
-                    "Results found on provider 'theia' is the total number of products "
-                    "available in the searched collection (e.g. SENTINEL2) instead of "
-                    "the total number of products matching the search criteria"
+            total_results = None if nb_res is None else total_results + nb_res
+            if count:
+                logger.info(
+                    "Found %s result(s) on provider '%s'",
+                    nb_res,
+                    search_plugin.provider,
                 )
+                # Hitting for instance
+                # https://theia.cnes.fr/atdistrib/resto2/api/collections/SENTINEL2/
+                #   search.json?startDate=2019-03-01&completionDate=2019-06-15
+                #   &processingLevel=LEVEL2A&maxRecords=1&page=1
+                # returns a number (properties.totalResults) that is the number of
+                # products in the collection (here SENTINEL2) instead of the estimated
+                # total number of products matching the search criteria (start/end date).
+                # Remove this warning when this is fixed upstream by THEIA.
+                if search_plugin.provider == "theia":
+                    logger.warning(
+                        "Results found on provider 'theia' is the total number of products "
+                        "available in the searched collection (e.g. SENTINEL2) instead of "
+                        "the total number of products matching the search criteria"
+                    )
         except Exception:
             logger.info(
                 "No result from provider '%s' due to an error during search. Raise "
                 "verbosity of log messages for details",
                 search_plugin.provider,
             )
-            if kwargs.get("raise_errors"):
+            if raise_errors:
                 # Raise the error, letting the application wrapping eodag know that
                 # something went bad. This way it will be able to decide what to do next
                 raise
@@ -819,7 +1091,7 @@ class EODataAccessGateway(object):
         progress_callback=None,
         wait=DEFAULT_DOWNLOAD_WAIT,
         timeout=DEFAULT_DOWNLOAD_TIMEOUT,
-        **kwargs
+        **kwargs,
     ):
         """Download all products resulting from a search.
 
@@ -857,7 +1129,7 @@ class EODataAccessGateway(object):
                 progress_callback=progress_callback,
                 wait=wait,
                 timeout=timeout,
-                **kwargs
+                **kwargs,
             )
             # close progress_bar when finished
             if hasattr(progress_callback, "pb") and hasattr(
@@ -923,7 +1195,7 @@ class EODataAccessGateway(object):
         provider=None,
         productType=None,
         timeout=HTTP_REQ_TIMEOUT,
-        **kwargs
+        **kwargs,
     ):
         """Loads STAC items from a geojson file / STAC catalog or collection, and convert to SearchResult.
 
@@ -992,7 +1264,7 @@ class EODataAccessGateway(object):
         progress_callback=None,
         wait=DEFAULT_DOWNLOAD_WAIT,
         timeout=DEFAULT_DOWNLOAD_TIMEOUT,
-        **kwargs
+        **kwargs,
     ):
         """Download a single product.
 
diff --git a/eodag/api/product/metadata_mapping.py b/eodag/api/product/metadata_mapping.py
index 7a50d2d5..2a00d1ac 100644
--- a/eodag/api/product/metadata_mapping.py
+++ b/eodag/api/product/metadata_mapping.py
@@ -22,8 +22,8 @@ from datetime import datetime
 from string import Formatter
 
 import geojson
-import jsonpath_ng as jsonpath
 from dateutil.tz import tzutc
+from jsonpath_ng.ext import parse
 from lxml import etree
 from lxml.etree import XPathEvalError
 from shapely import wkt
@@ -431,7 +431,7 @@ def properties_from_json(json, mapping, discovery_pattern=None, discovery_path=N
 
     # adds missing discovered properties
     if discovery_pattern and discovery_path:
-        discovered_properties = jsonpath.parse(discovery_path).find(json)
+        discovered_properties = parse(discovery_path).find(json)
         for found_jsonpath in discovered_properties:
             found_key = found_jsonpath.path.fields[-1]
             if (
@@ -592,9 +592,9 @@ def mtd_cfg_as_jsonpath(src_dict, dest_dict={}):
             try:
                 # If the metadata is queryable (i.e a list of 2 elements), replace the value of the last item
                 if len(dest_dict[metadata]) == 2:
-                    dest_dict[metadata][1] = (conversion, jsonpath.parse(path))
+                    dest_dict[metadata][1] = (conversion, parse(path))
                 else:
-                    dest_dict[metadata] = (conversion, jsonpath.parse(path))
+                    dest_dict[metadata] = (conversion, parse(path))
             except Exception:  # jsonpath_ng does not provide a proper exception
                 # Keep path as this and its associated conversion
                 if len(dest_dict[metadata]) == 2:
diff --git a/eodag/cli.py b/eodag/cli.py
index b25bfa6b..9832e1c7 100755
--- a/eodag/cli.py
+++ b/eodag/cli.py
@@ -210,11 +210,12 @@ def version():
 @click.option(
     "--items",
     type=int,
-    default=DEFAULT_ITEMS_PER_PAGE,
-    show_default=True,
+    # default=DEFAULT_ITEMS_PER_PAGE,
+    show_default=False,
     help="The number of items to return. Eodag is bound to whatever limitation the "
     "providers have on the number of results they return. This option allows "
-    "to control how many items eodag should request",
+    "to control how many items eodag should request "
+    f"[default: {DEFAULT_ITEMS_PER_PAGE}]",
 )
 @click.option(
     "--page",
@@ -223,6 +224,15 @@ def version():
     show_default=True,
     help="Retrieve the given page",
 )
+@click.option(
+    "--all",
+    is_flag=True,
+    help="Retrieve ALL the products that match the search criteria. It collects "
+    "products by iterating over the results pages until no more products are available."
+    "At each iteration, the maximum number of items searched is either 'items' if set, "
+    "or a maximum value defined internally for the requested provider, or a default "
+    "maximum value equals to 50.",
+)
 @click.option(
     "--locations",
     type=str,
@@ -264,8 +274,7 @@ def search_crunch(ctx, **kwargs):
             click.echo(search_crunch.get_help(ctx))
         sys.exit(-1)
 
-    kwargs["verbose"] = ctx.obj["verbosity"]
-    setup_logging(**kwargs)
+    setup_logging(verbose=ctx.obj["verbosity"])
 
     if kwargs["box"] != (None,) * 4:
         rect = kwargs.pop("box")
@@ -333,10 +342,21 @@ def search_crunch(ctx, **kwargs):
     )
 
     # Search
-    results, total = gateway.search(
-        page=page, items_per_page=items_per_page, **criteria
-    )
-    click.echo("Found a total number of {} products".format(total))
+    get_all_products = kwargs.pop("all")
+    if get_all_products:
+        # search_all needs items_per_page to be None if the user lets eodag determines
+        # what value it should take.
+        items_per_page = None if items_per_page is None else items_per_page
+        results = gateway.search_all(items_per_page=items_per_page, **criteria)
+    else:
+        # search should better take a value that is not None
+        items_per_page = (
+            DEFAULT_ITEMS_PER_PAGE if items_per_page is None else items_per_page
+        )
+        results, total = gateway.search(
+            page=page, items_per_page=items_per_page, **criteria
+        )
+        click.echo("Found a total number of {} products".format(total))
     click.echo("Returned {} products".format(len(results)))
 
     # Crunch !
@@ -384,7 +404,7 @@ def list_pt(ctx, **kwargs):
             platformSerialIdentifier=kwargs.get("platformserialidentifier"),
             processingLevel=kwargs.get("processinglevel"),
             sensorType=kwargs.get("sensortype"),
-            **kwargs
+            **kwargs,
         )
     except NoMatchingProductType:
         if any(
@@ -457,8 +477,7 @@ def download(ctx, **kwargs):
             click.echo("Nothing to do (no search results file provided)")
             click.echo(download.get_help(ctx))
         sys.exit(1)
-    kwargs["verbose"] = ctx.obj["verbosity"]
-    setup_logging(**kwargs)
+    setup_logging(verbose=ctx.obj["verbosity"])
     conf_file = kwargs.pop("conf")
     if conf_file:
         conf_file = click.format_filename(conf_file)
diff --git a/eodag/config.py b/eodag/config.py
index 2a9e45d7..2334ce75 100644
--- a/eodag/config.py
+++ b/eodag/config.py
@@ -335,6 +335,38 @@ def override_config_from_mapping(config, mapping):
                 logger.warning("%s skipped: %s", provider, e)
 
 
+def merge_configs(config, other_config):
+    """Override a configuration with the values of another configuration
+
+    :param dict config: An eodag providers configuration dictionary
+    :param dict other_config: An eodag providers configuration dictionary
+    """
+    # configs union with other_config values as default
+    other_config = dict(config, **other_config)
+
+    for provider, new_conf in other_config.items():
+        old_conf = config.get(provider, None)
+
+        if old_conf:
+            # update non-objects values
+            new_conf = dict(old_conf.__dict__, **new_conf.__dict__)
+
+            for conf_k, conf_v in new_conf.items():
+                old_conf_v = getattr(old_conf, conf_k, None)
+
+                if isinstance(conf_v, PluginConfig) and isinstance(
+                    old_conf_v, PluginConfig
+                ):
+                    old_conf_v.update(conf_v.__dict__)
+                    new_conf[conf_k] = old_conf_v
+                elif isinstance(old_conf_v, PluginConfig):
+                    new_conf[conf_k] = old_conf_v
+
+                setattr(config[provider], conf_k, new_conf[conf_k])
+        else:
+            config[provider] = new_conf
+
+
 def load_yml_config(yml_path):
     """Build a conf dictionnary from given yml absolute path
 
diff --git a/eodag/plugins/apis/usgs.py b/eodag/plugins/apis/usgs.py
index cadf9249..37ce4443 100644
--- a/eodag/plugins/apis/usgs.py
+++ b/eodag/plugins/apis/usgs.py
@@ -16,16 +16,13 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 import copy
-import hashlib
 import logging
-import os
-import re
-import zipfile
+import shutil
+import tarfile
 
 import requests
+from jsonpath_ng.ext import parse
 from requests import HTTPError
-from shapely import geometry
-from tqdm import tqdm
 from usgs import USGSError, api
 
 from eodag.api.product import EOProduct
@@ -35,17 +32,31 @@ from eodag.api.product.metadata_mapping import (
     properties_from_json,
 )
 from eodag.plugins.apis.base import Api
+from eodag.plugins.download.base import (
+    DEFAULT_DOWNLOAD_TIMEOUT,
+    DEFAULT_DOWNLOAD_WAIT,
+    Download,
+)
+from eodag.utils import GENERIC_PRODUCT_TYPE, format_dict_items, get_progress_callback
 from eodag.utils.exceptions import AuthenticationError, NotAvailableError
 
 logger = logging.getLogger("eodag.plugins.apis.usgs")
 
 
-class UsgsApi(Api):
+class UsgsApi(Api, Download):
     """A plugin that enables to query and download data on the USGS catalogues"""
 
-    def query(self, product_type=None, count=True, **kwargs):
+    def query(
+        self, product_type=None, items_per_page=None, page=None, count=True, **kwargs
+    ):
         """Search for data on USGS catalogues
 
+        .. versionchanged::
+           2.2.0
+
+                * Based on usgs library v0.3.0 which now uses M2M API. The library
+                  is used for both search & download
+
         .. versionchanged::
             1.0
 
@@ -62,8 +73,11 @@ class UsgsApi(Api):
             )
         except USGSError:
             raise AuthenticationError("Please check your USGS credentials.") from None
-        usgs_dataset = self.config.products[product_type]["dataset"]
-        usgs_catalog_node = self.config.products[product_type]["catalog_node"]
+
+        product_type_def_params = self.config.products.get(
+            product_type, self.config.products[GENERIC_PRODUCT_TYPE]
+        )
+        usgs_dataset = format_dict_items(product_type_def_params, **kwargs)["dataset"]
         start_date = kwargs.pop("startTimeFromAscendingNode", None)
         end_date = kwargs.pop("completionTimeFromAscendingNode", None)
         geom = kwargs.pop("geometry", None)
@@ -78,16 +92,6 @@ class UsgsApi(Api):
         else:
             footprint = geom
 
-        # Configuration to generate the download url of search results
-        result_summary_pattern = re.compile(
-            r"^ID: .+, Acquisition Date: .+, Path: (?P<path>\d+), Row: (?P<row>\d+)$"  # noqa
-        )
-        # See https://pyformat.info/, on section "Padding and aligning strings" to
-        # understand {path:0>3} and {row:0>3}.
-        # It roughly means: 'if the string that will be passed as "path" has length < 3,
-        # prepend as much "0"s as needed to reach length 3' and same for "row"
-        dl_url_pattern = "{base_url}/L8/{path:0>3}/{row:0>3}/{entity}.tar.bz"
-
         final = []
         if footprint and len(footprint.keys()) == 4:  # a rectangle (or bbox)
             lower_left = {
@@ -101,51 +105,33 @@ class UsgsApi(Api):
         else:
             lower_left, upper_right = None, None
         try:
-            results = api.search(
+            results = api.scene_search(
                 usgs_dataset,
-                usgs_catalog_node,
                 start_date=start_date,
                 end_date=end_date,
                 ll=lower_left,
                 ur=upper_right,
+                max_results=items_per_page,
+                starting_number=(1 + (page - 1) * items_per_page),
             )
 
-            for result in results["data"]["results"]:
-                r_lower_left = result["spatialFootprint"]["coordinates"][0][0]
-                r_upper_right = result["spatialFootprint"]["coordinates"][0][2]
-                summary_match = result_summary_pattern.match(
-                    result["summary"]
-                ).groupdict()
-                result["geometry"] = geometry.box(
-                    r_lower_left[0], r_lower_left[1], r_upper_right[0], r_upper_right[1]
-                )
+            # Same method as in base.py, Search.__init__()
+            # Prepare the metadata mapping
+            # Do a shallow copy, the structure is flat enough for this to be sufficient
+            metas = DEFAULT_METADATA_MAPPING.copy()
+            # Update the defaults with the mapping value. This will add any new key
+            # added by the provider mapping that is not in the default metadata.
+            # A deepcopy is done to prevent self.config.metadata_mapping from being modified when metas[metadata]
+            # is a list and is modified
+            metas.update(copy.deepcopy(self.config.metadata_mapping))
+            metas = mtd_cfg_as_jsonpath(metas)
 
-                # Same method as in base.py, Search.__init__()
-                # Prepare the metadata mapping
-                # Do a shallow copy, the structure is flat enough for this to be sufficient
-                metas = DEFAULT_METADATA_MAPPING.copy()
-                # Update the defaults with the mapping value. This will add any new key
-                # added by the provider mapping that is not in the default metadata.
-                # A deepcopy is done to prevent self.config.metadata_mapping from being modified when metas[metadata]
-                # is a list and is modified
-                metas.update(copy.deepcopy(self.config.metadata_mapping))
-                metas = mtd_cfg_as_jsonpath(metas)
+            for result in results["data"]["results"]:
 
                 result["productType"] = usgs_dataset
 
                 product_properties = properties_from_json(result, metas)
 
-                if getattr(self.config, "product_location_scheme", "https") == "file":
-                    product_properties["downloadLink"] = dl_url_pattern.format(
-                        base_url="file://"
-                    )
-                else:
-                    product_properties["downloadLink"] = dl_url_pattern.format(
-                        base_url=self.config.google_base_url.rstrip("/"),
-                        entity=result["entityId"],
-                        **summary_match
-                    )
-
                 final.append(
                     EOProduct(
                         productType=product_type,
@@ -155,97 +141,149 @@ class UsgsApi(Api):
                     )
                 )
         except USGSError as e:
-            logger.debug(
-                "Product type %s does not exist on catalogue %s",
+            logger.warning(
+                "Product type %s does not exist on USGS EE catalog",
                 usgs_dataset,
-                usgs_catalog_node,
             )
-            logger.debug("Skipping error: %s", e)
+            logger.warning("Skipping error: %s", e)
         api.logout()
-        total_results = len(final) if count else None
+
+        if final:
+            # parse total_results
+            path_parsed = parse(self.config.pagination["total_items_nb_key_path"])
+            total_results = path_parsed.find(results["data"])[0].value
+        else:
+            total_results = 0
+
         return final, total_results
 
     def download(self, product, auth=None, progress_callback=None, **kwargs):
         """Download data from USGS catalogues"""
-        url = product.remote_location
-        if not url:
-            logger.debug(
-                "Unable to get download url for %s, skipping download", product
+
+        fs_path, record_filename = self._prepare_download(
+            product, outputs_extension=".tar.gz", **kwargs
+        )
+        if not fs_path or not record_filename:
+            return fs_path
+
+        # progress bar init
+        if progress_callback is None:
+            progress_callback = get_progress_callback()
+        progress_callback.desc = product.properties.get("id", "")
+        progress_callback.position = 1
+
+        try:
+            api.login(
+                self.config.credentials["username"],
+                self.config.credentials["password"],
+                save=True,
             )
-            return
-        logger.info("Download url: %s", url)
+        except USGSError:
+            raise AuthenticationError("Please check your USGS credentials.") from None
 
-        filename = product.properties["title"] + ".tar.bz"
-        outputs_prefix = (
-            kwargs.pop("outputs_prefix", None) or self.config.outputs_prefix
+        download_options = api.download_options(
+            product.properties["productType"], product.properties["id"]
         )
-        local_file_path = os.path.join(outputs_prefix, filename)
-        download_records = os.path.join(outputs_prefix, ".downloaded")
-        if not os.path.exists(download_records):
-            os.makedirs(download_records)
-        url_hash = hashlib.md5(url.encode("utf-8")).hexdigest()
-        record_filename = os.path.join(download_records, url_hash)
-        if os.path.isfile(record_filename) and os.path.isfile(local_file_path):
-            logger.info(
-                "Product already downloaded. Retrieve it at %s", local_file_path
+
+        try:
+            product_ids = [
+                p["id"]
+                for p in download_options["data"]
+                if p["downloadSystem"] == "dds"
+            ]
+        except KeyError as e:
+            raise NotAvailableError(
+                "%s not found in %s's products" % (e, product.properties["id"])
             )
-            return local_file_path
-        # Remove the record file if local_file_path is absent (e.g. it was deleted
-        # while record wasn't)
-        elif os.path.isfile(record_filename):
-            logger.debug(
-                "Record file found (%s) but not the actual file", record_filename
+
+        if not product_ids:
+            raise NotAvailableError(
+                "No USGS products found for %s" % product.properties["id"]
             )
-            logger.debug("Removing record file : %s", record_filename)
-            os.remove(record_filename)
-        params = kwargs.pop("dl_url_params", None) or getattr(
-            self.config, "dl_url_params", {}
-        )
+
+        req_urls = []
+        for product_id in product_ids:
+            download_request = api.download_request(
+                product.properties["productType"], product.properties["id"], product_id
+            )
+            try:
+                req_urls.extend(
+                    [x["url"] for x in download_request["data"]["preparingDownloads"]]
+                )
+            except KeyError as e:
+                raise NotAvailableError(
+                    "%s not found in %s download_request"
+                    % (e, product.properties["id"])
+                )
+
+        if len(req_urls) > 1:
+            logger.warning(
+                "%s usgs products found for %s. Only first will be downloaded"
+                % (len(req_urls), product.properties["id"])
+            )
+        elif not req_urls:
+            raise NotAvailableError(
+                "No usgs request url was found for %s" % product.properties["id"]
+            )
+
+        req_url = req_urls[0]
+        progress_callback.reset()
         with requests.get(
-            url,
+            req_url,
             stream=True,
-            auth=auth,
-            params=params,
-            verify=False,
-            hooks={"response": lambda r, *args, **kwargs: print("\n", r.url)},
         ) as stream:
-            stream_size = int(stream.headers.get("content-length", 0))
-            with open(local_file_path, "wb") as fhandle:
-                for chunk in stream.iter_content(chunk_size=64 * 1024):
-                    if chunk:
-                        fhandle.write(chunk)
-                        progress_callback(len(chunk), stream_size)
             try:
                 stream.raise_for_status()
-            except HTTPError as e:
-                if e.response.status_code == 404:
-                    raise NotAvailableError(
-                        "%s not available, request returned: %s"
-                        % (product.properties["title"], e)
-                    )
-                else:
-                    import traceback
+            except HTTPError:
+                import traceback as tb
 
-                    logger.error(
-                        "Error while getting resource : %s", traceback.format_exc()
-                    )
+                logger.error(
+                    "Error while getting resource :\n%s",
+                    tb.format_exc(),
+                )
             else:
-                with open(record_filename, "w") as fh:
-                    fh.write(url)
-                logger.debug("Download recorded in %s", record_filename)
-                extract = kwargs.pop("extract", None)
-                extract = extract if extract is not None else self.config.extract
-                if extract and zipfile.is_zipfile(local_file_path):
-                    logger.info("Extraction activated")
-                    with zipfile.ZipFile(local_file_path, "r") as zfile:
-                        fileinfos = zfile.infolist()
-                        with tqdm(
-                            fileinfos,
-                            unit="file",
-                            desc="Extracting files from {}".format(local_file_path),
-                        ) as progressbar:
-                            for fileinfo in progressbar:
-                                zfile.extract(fileinfo, path=outputs_prefix)
-                    return local_file_path[: local_file_path.index(".tar.bz")]
-                else:
-                    return local_file_path
+                stream_size = int(stream.headers.get("content-length", 0))
+                progress_callback.max_size = stream_size
+                progress_callback.reset()
+                with open(fs_path, "wb") as fhandle:
+                    for chunk in stream.iter_content(chunk_size=64 * 1024):
+                        if chunk:
+                            fhandle.write(chunk)
+                            progress_callback(len(chunk), stream_size)
+
+        with open(record_filename, "w") as fh:
+            fh.write(product.properties["downloadLink"])
+        logger.debug("Download recorded in %s", record_filename)
+
+        api.logout()
+
+        # Check that the downloaded file is really a tar file
+        if not tarfile.is_tarfile(fs_path):
+            logger.warning(
+                "Downloaded product is not a tar File. Please check its file type before using it"
+            )
+            new_fs_path = fs_path[: fs_path.index(".tar.gz")]
+            shutil.move(fs_path, new_fs_path)
+            return new_fs_path
+        return self._finalize(fs_path, outputs_extension=".tar.gz", **kwargs)
+
+    def download_all(
+        self,
+        products,
+        auth=None,
+        progress_callback=None,
+        wait=DEFAULT_DOWNLOAD_WAIT,
+        timeout=DEFAULT_DOWNLOAD_TIMEOUT,
+        **kwargs
+    ):
+        """
+        download_all using parent (base plugin) method
+        """
+        return super(UsgsApi, self).download_all(
+            products,
+            auth=auth,
+            progress_callback=progress_callback,
+            wait=wait,
+            timeout=timeout,
+            **kwargs
+        )
diff --git a/eodag/plugins/download/base.py b/eodag/plugins/download/base.py
index 75427834..8d6dbd3e 100644
--- a/eodag/plugins/download/base.py
+++ b/eodag/plugins/download/base.py
@@ -19,6 +19,7 @@
 import hashlib
 import logging
 import os
+import tarfile
 import zipfile
 from datetime import datetime, timedelta
 from time import sleep
@@ -98,6 +99,8 @@ class Download(PluginTopic):
         outputs_prefix = (
             kwargs.pop("outputs_prefix", None) or self.config.outputs_prefix
         )
+        outputs_extension = kwargs.get("outputs_extension", ".zip")
+
         # Strong asumption made here: all products downloaded will be zip files
         # If they are not, the '.zip' extension will be removed when they are downloaded and returned as is
         prefix = os.path.abspath(outputs_prefix)
@@ -108,11 +111,13 @@ class Download(PluginTopic):
             collision_avoidance_suffix = "-" + sanitize(product.properties["id"])
         fs_path = os.path.join(
             prefix,
-            "{}{}.zip".format(
-                sanitize(product.properties["title"]), collision_avoidance_suffix
+            "{}{}{}".format(
+                sanitize(product.properties["title"]),
+                collision_avoidance_suffix,
+                outputs_extension,
             ),
         )
-        fs_dir_path = fs_path.replace(".zip", "")
+        fs_dir_path = fs_path.replace(outputs_extension, "")
         download_records_dir = os.path.join(prefix, ".downloaded")
         try:
             os.makedirs(download_records_dir)
@@ -154,11 +159,15 @@ class Download(PluginTopic):
         extract = (
             extract if extract is not None else getattr(self.config, "extract", False)
         )
+        outputs_extension = kwargs.pop("outputs_extension", ".zip")
+
         if not extract:
             logger.info("Extraction not activated. The product is available as is.")
             return fs_path
         product_path = (
-            fs_path[: fs_path.index(".zip")] if ".zip" in fs_path else fs_path
+            fs_path[: fs_path.index(outputs_extension)]
+            if outputs_extension in fs_path
+            else fs_path
         )
         product_path_exists = os.path.exists(product_path)
         if product_path_exists and os.path.isfile(product_path):
@@ -195,22 +204,30 @@ class Download(PluginTopic):
         )
         if not os.path.exists(product_path):
             logger.info("Extraction activated")
-            with zipfile.ZipFile(fs_path, "r") as zfile:
-                fileinfos = zfile.infolist()
-                with get_progress_callback() as bar:
-                    bar.max_size = len(fileinfos)
-                    bar.unit = "file"
-                    bar.desc = "Extracting files from {}".format(
-                        os.path.basename(fs_path)
-                    )
-                    bar.unit_scale = False
-                    bar.position = 2
-                    for fileinfo in fileinfos:
-                        zfile.extract(
-                            fileinfo,
-                            path=os.path.join(outputs_prefix, product_path),
+            if fs_path.endswith(".zip"):
+                with zipfile.ZipFile(fs_path, "r") as zfile:
+                    fileinfos = zfile.infolist()
+                    with get_progress_callback() as bar:
+                        bar.max_size = len(fileinfos)
+                        bar.unit = "file"
+                        bar.desc = "Extracting files from {}".format(
+                            os.path.basename(fs_path)
                         )
-                        bar(1)
+                        bar.unit_scale = False
+                        bar.position = 2
+                        for fileinfo in fileinfos:
+                            zfile.extract(
+                                fileinfo,
+                                path=os.path.join(outputs_prefix, product_path),
+                            )
+                            bar(1)
+            elif fs_path.endswith(".tar.gz"):
+                with tarfile.open(fs_path, "r:gz") as zfile:
+                    logger.info(
+                        "Extracting files from {}".format(os.path.basename(fs_path))
+                    )
+                    zfile.extractall(path=os.path.join(outputs_prefix, product_path))
+
         # Handle depth levels in the product archive. For example, if the downloaded archive was
         # extracted to: /top_level/product_base_dir and archive_depth was configured to 2, the product
         # location will be /top_level/product_base_dir.
diff --git a/eodag/plugins/manager.py b/eodag/plugins/manager.py
index 8abf723b..b2c906be 100644
--- a/eodag/plugins/manager.py
+++ b/eodag/plugins/manager.py
@@ -21,7 +21,7 @@ from pathlib import Path
 
 import pkg_resources
 
-from eodag.config import load_config
+from eodag.config import load_config, merge_configs
 from eodag.plugins.apis.base import Api
 from eodag.plugins.authentication.base import Authentication
 from eodag.plugins.base import EODAGPluginMount
@@ -79,20 +79,33 @@ class PluginManager(object):
                         "Check that the plugin module (%s) is importable",
                         entry_point.module_name,
                     )
-                if entry_point.dist.project_name != "eodag":
+                if entry_point.dist.key != "eodag":
                     # use plugin providers if any
                     plugin_providers_config_path = [
                         x
-                        for x in Path(entry_point.dist.location).rglob("providers.yml")
-                        if all(i not in str(x) for i in ["build", ".tox"])
+                        for x in Path(
+                            entry_point.dist.location,
+                            pkg_resources.to_filename(entry_point.dist.key),
+                        ).rglob("providers.yml")
                     ]
                     if plugin_providers_config_path:
-                        self.providers_config.update(
-                            load_config(plugin_providers_config_path[0])
+                        plugin_providers_config = load_config(
+                            plugin_providers_config_path[0]
                         )
+                        merge_configs(plugin_providers_config, self.providers_config)
+                        self.providers_config = plugin_providers_config
 
         self.product_type_to_provider_config_map = {}
-        for provider_config in self.providers_config.values():
+        for provider in list(self.providers_config):
+            provider_config = self.providers_config[provider]
+            if not hasattr(provider_config, "products"):
+                logger.info(
+                    "%s: provider has no product configured and will be skipped"
+                    % provider
+                )
+                providers_config.pop(provider)
+                continue
+
             for product_type in provider_config.products:
                 product_type_providers = (
                     self.product_type_to_provider_config_map.setdefault(  # noqa
diff --git a/eodag/plugins/search/qssearch.py b/eodag/plugins/search/qssearch.py
index dbad8b6e..b238440a 100644
--- a/eodag/plugins/search/qssearch.py
+++ b/eodag/plugins/search/qssearch.py
@@ -22,8 +22,8 @@ import re
 from urllib.error import HTTPError as urllib_HTTPError
 from urllib.request import urlopen
 
-import jsonpath_ng as jsonpath
 import requests
+from jsonpath_ng.ext import parse
 from lxml import etree
 
 from eodag.api.product import EOProduct
@@ -92,6 +92,9 @@ class QueryStringSearch(Search):
           - *count_endpoint*: (optional) The endpoint for counting the number of items
             satisfying a request
 
+            *next_page_url_key_path: (optional) A JSONPATH expression used to retrieve
+            the URL of the next page in the response of the current page.
+
         - **free_text_search_operations**: (optional) A tree structure of the form::
 
             <search-param>:     # e.g: $search
@@ -153,9 +156,19 @@ class QueryStringSearch(Search):
         self.search_urls = []
         self.query_params = dict()
         self.query_string = ""
+        self.next_page_url = None
 
     def query(self, items_per_page=None, page=None, count=True, **kwargs):
-        """Perform a search on an OpenSearch-like interface"""
+        """Perform a search on an OpenSearch-like interface
+
+        :param page: The page number to retur (default: 1)
+        :type page: int
+        :param items_per_page: The number of results that must appear in one single
+                               page
+        :type items_per_page: int
+        :param count:  To trigger a count request (default: True)
+        :type count: bool
+        """
         product_type = kwargs.get("productType", None)
         if product_type == GENERIC_PRODUCT_TYPE:
             logger.warning(
@@ -417,6 +430,9 @@ class QueryStringSearch(Search):
             except RequestError:
                 raise StopIteration
             else:
+                next_page_url_key_path = self.config.pagination.get(
+                    "next_page_url_key_path"
+                )
                 if self.config.result_type == "xml":
                     root_node = etree.fromstring(response.content)
                     namespaces = {k or "ns": v for k, v in root_node.nsmap.items()}
@@ -426,8 +442,24 @@ class QueryStringSearch(Search):
                             self.config.results_entry, namespaces=namespaces
                         )
                     ]
+                    if next_page_url_key_path:
+                        raise NotImplementedError(
+                            "Setting the next page url from an XML response has not "
+                            "been implemented yet"
+                        )
                 else:
-                    result = response.json().get(self.config.results_entry, [])
+                    resp_as_json = response.json()
+                    if next_page_url_key_path:
+                        path_parsed = parse(next_page_url_key_path)
+                        try:
+                            self.next_page_url = path_parsed.find(resp_as_json)[0].value
+                            logger.debug(
+                                "Next page URL collected and set for the next search",
+                            )
+                        except IndexError:
+                            logger.debug("Next page URL could not be collected")
+
+                    result = resp_as_json.get(self.config.results_entry, [])
                 if getattr(self.config, "merge_responses", False):
                     results = (
                         [dict(r, **result[i]) for i, r in enumerate(results)]
@@ -490,9 +522,7 @@ class QueryStringSearch(Search):
         else:
             count_results = response.json()
             if isinstance(count_results, dict):
-                path_parsed = jsonpath.parse(
-                    self.config.pagination["total_items_nb_key_path"]
-                )
+                path_parsed = parse(self.config.pagination["total_items_nb_key_path"])
                 total_results = path_parsed.find(count_results)[0].value
             else:  # interpret the result as a raw int
                 total_results = int(count_results)
@@ -730,12 +760,12 @@ class PostJsonSearch(QueryStringSearch):
                     if len(self.config.metadata_mapping[metadata]) == 2:
                         self.config.metadata_mapping[metadata][1] = (
                             conversion,
-                            jsonpath.parse(path),
+                            parse(path),
                         )
                     else:
                         self.config.metadata_mapping[metadata] = (
                             conversion,
-                            jsonpath.parse(path),
+                            parse(path),
                         )
                 except Exception:  # jsonpath_ng does not provide a proper exception
                     # Assume the mapping is to be passed as is.
diff --git a/eodag/resources/providers.yml b/eodag/resources/providers.yml
index c034296e..01727572 100644
--- a/eodag/resources/providers.yml
+++ b/eodag/resources/providers.yml
@@ -26,23 +26,29 @@
   api: !plugin
     type: UsgsApi
     google_base_url: 'http://storage.googleapis.com/earthengine-public/landsat/'
+    pagination:
+      max_items_per_page: 5000
+      total_items_nb_key_path: '$.totalHits'
     metadata_mapping:
       id: '$.entityId'
-      geometry: '$.geometry'
+      geometry: '$.spatialBounds'
       productType: '$.productType'
       title: '$.displayId'
       abstract: '$.summary'
-      startTimeFromAscendingNode: '$.startTime'
-      completionTimeFromAscendingNode: '$.endTime'
-      quicklook: '$.browseUrl'
+      cloudCover: '$.cloudCover'
+      startTimeFromAscendingNode: '$.temporalCoverage.startDate'
+      completionTimeFromAscendingNode: '$.temporalCoverage.endDate'
+      quicklook: '$.browse[0].thumbnailPath'
       # storageStatus set to ONLINE for consistency between providers
       storageStatus: '{$.null#replace_str("Not Available","ONLINE")}'
+      downloadLink: 'https://dds.cr.usgs.gov/{id}'
     extract: True
   products:
     # datasets list http://kapadia.github.io/usgs/_sources/reference/catalog/ee.txt
     L8_OLI_TIRS_C1L1:
       dataset: LANDSAT_8_C1
-      catalog_node: EE  # Possible values: CWIC, EE, HDDS, LPCS
+    GENERIC_PRODUCT_TYPE:
+      dataset: '{productType}'
 
 ---
 !provider
@@ -504,6 +510,8 @@
     pagination:
       next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&page={page}'
       total_items_nb_key_path: '$.properties.totalResults'
+      # 2019/03/19: Returns a 400 error code if greater than 500.
+      max_items_per_page: 500
     discover_metadata:
       auto_discovery: true
       metadata_pattern: '^(?!collection)[a-zA-Z0-9_]+$'
@@ -676,6 +684,8 @@
     pagination:
       next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&page={page}'
       total_items_nb_key_path: '$.properties.totalResults'
+      # 2019/03/19: 500 is the max, no error if greater
+      max_items_per_page: 500
     discover_metadata:
       auto_discovery: true
       metadata_pattern: '^(?!collection)[a-zA-Z0-9_]+$'
@@ -866,6 +876,9 @@
     pagination:
       next_page_url_tpl: '{url}?{search}&size={items_per_page}&from={skip}'
       total_items_nb_key_path: '$.totalnb'
+      # 2019/03/19: It's not possible to get more than 10_000 products
+      # Returns a 400 Bad Request if more products are requested.
+      max_items_per_page: 10_000
     results_entry: 'hits'
     discover_metadata:
       auto_discovery: true
@@ -953,6 +966,8 @@
     pagination:
       next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&page={page}'
       total_items_nb_key_path: '$.properties.totalResults'
+      # 2019/03/19: 2000 is the max, 400 error if greater
+      max_items_per_page: 2_000
     discover_metadata:
       auto_discovery: true
       metadata_pattern: '^(?!collection)[a-zA-Z0-9]+$'
@@ -1106,6 +1121,8 @@
     pagination:
       next_page_url_tpl: '{url}?{search}&maxRecords={items_per_page}&startIndex={skip_base_1}'
       total_items_nb_key_path: '//os:totalResults/text()'
+      # 2019/03/19: 50 is the max, no error if greater
+      max_items_per_page: 50
     discover_metadata:
       auto_discovery: true
       metadata_pattern: '^(?!collection)[a-zA-Z0-9]+$'
@@ -1328,6 +1345,8 @@
     pagination:
       count_endpoint: 'https://catalogue.onda-dias.eu/dias-catalogue/Products/$count'
       next_page_url_tpl: '{url}?{search}&$top={items_per_page}&$skip={skip}'
+      # 2019/03/19: 2000 is the max, if greater 200 response but contains an error message
+      max_items_per_page: 2_000
     results_entry: 'value'
     literal_search_params:
       $format: json
@@ -1463,6 +1482,13 @@
   search: !plugin
     type: StacSearch
     api_endpoint: https://eod-catalog-svc-prod.astraea.earth/search
+    pagination:
+      # 2019/03/19: The docs (https://eod-catalog-svc-prod.astraea.earth/api.html#operation/getSearchSTAC)
+      # say the max is 10_000. In practice 1_000 products are returned if more are asked (even greater
+      # than 10_000), without any error.
+      # This provider doesn't implement any pagination, let's just try to get the maximum number of
+      # products available at once then, so we stick to 10_000.
+      max_items_per_page: 10_000
   products:
     S2_MSI_L1C:
       productType: sentinel2_l1c
@@ -1496,6 +1522,10 @@
     api_endpoint: https://landsatlook.usgs.gov/sat-api/collections/{collection}/items
     pagination:
       total_items_nb_key_path: '$.meta.found'
+      # 2019/03/19: no more than 10_000 (if greater, returns a 500 error code)
+      # but in practive if an Internal Server Error is returned for more than
+      # about 500 products.
+      max_items_per_page: 500
     metadata_mapping:
       productType: '$.collection'
       completionTimeFromAscendingNode:
diff --git a/eodag/resources/stac_provider.yml b/eodag/resources/stac_provider.yml
index d9529b32..b6f2ed19 100644
--- a/eodag/resources/stac_provider.yml
+++ b/eodag/resources/stac_provider.yml
@@ -20,6 +20,7 @@ search:
   pagination:
     next_page_url_tpl: '{url}?{search}&limit={items_per_page}&page={page}'
     total_items_nb_key_path: '$.context.matched'
+    next_page_url_key_path: '$.links[?(@.rel="next")].href'
   discover_metadata:
     auto_discovery: true
     metadata_pattern: '^[a-zA-Z0-9_:-]+$'
diff --git a/eodag/utils/__init__.py b/eodag/utils/__init__.py
index 2570c20c..e1ec7d9d 100644
--- a/eodag/utils/__init__.py
+++ b/eodag/utils/__init__.py
@@ -44,9 +44,10 @@ from urllib.parse import (  # noqa; noqa
 )
 
 import click
-import jsonpath_ng as jsonpath
 import shapefile
 import shapely.wkt
+from jsonpath_ng import jsonpath
+from jsonpath_ng.ext import parse
 from requests.auth import AuthBase
 from shapely.geometry import Polygon, shape
 from shapely.geometry.base import BaseGeometry
@@ -447,9 +448,9 @@ def format_dict_items(config_dict, **format_variables):
 def jsonpath_parse_dict_items(jsonpath_dict, values_dict):
     """Recursive parse jsonpath elements in dict
 
-    >>> import jsonpath_ng as jsonpath
+    >>> import jsonpath_ng.ext as jsonpath
     >>> jsonpath_parse_dict_items(
-    ...     {"foo": {"bar": jsonpath.parse("$.a.b")}, "qux": [jsonpath.parse("$.c"), jsonpath.parse("$.c")]},
+    ...     {"foo": {"bar": parse("$.a.b")}, "qux": [parse("$.c"), parse("$.c")]},
     ...     {"a":{"b":"baz"}, "c":"quux"}
     ... ) == {'foo': {'bar': 'baz'}, 'qux': ['quux', 'quux']}
     True
@@ -635,7 +636,7 @@ def string_to_jsonpath(key, str_value):
     """
     if "$." in str(str_value):
         try:
-            return jsonpath.parse(str_value)
+            return parse(str_value)
         except Exception:  # jsonpath_ng does not provide a proper exception
             # If str_value does not contain a jsonpath, return it as is
             return str_value
@@ -684,8 +685,8 @@ def format_string(key, str_to_format, **format_variables):
 def parse_jsonpath(key, jsonpath_obj, **values_dict):
     """Parse jsonpah in jsonpath_obj using values_dict
 
-    >>> import jsonpath_ng as jsonpath
-    >>> parse_jsonpath(None, jsonpath.parse("$.foo.bar"), **{"foo":{"bar":"baz"}})
+    >>> import jsonpath_ng.ext as jsonpath
+    >>> parse_jsonpath(None, parse("$.foo.bar"), **{"foo":{"bar":"baz"}})
     'baz'
 
     :param key: input item key
@@ -697,7 +698,7 @@ def parse_jsonpath(key, jsonpath_obj, **values_dict):
     :returns: parsed value
     :rtype: str
     """
-    if isinstance(jsonpath_obj, jsonpath.jsonpath.Child):
+    if isinstance(jsonpath_obj, jsonpath.Child):
         match = jsonpath_obj.find(values_dict)
         return match[0].value if len(match) == 1 else None
     else:
diff --git a/eodag/utils/logging.py b/eodag/utils/logging.py
index 2a6ed5aa..12e629ea 100644
--- a/eodag/utils/logging.py
+++ b/eodag/utils/logging.py
@@ -19,15 +19,17 @@
 import logging.config
 
 
-def setup_logging(**kwargs):
+def setup_logging(verbose):
     """Define logging level
 
-    :param kwargs: Accepted Keyword arguments:
-                    verbose=0-3 logging level (None to max)
-    :type kwargs: dict
+    :param verbose: Accepted values:
+
+                    * 0: no logging
+                    * 1 or 2: INFO level
+                    * 3: DEBUG level
+    :type verbose: int
     """
-    verbosity = kwargs.pop("verbose")
-    if verbosity == 0:
+    if verbose == 0:
         logging.config.dictConfig(
             {
                 "version": 1,
@@ -40,7 +42,7 @@ def setup_logging(**kwargs):
                 },
             }
         )
-    elif verbosity <= 2:
+    elif verbose <= 2:
         logging.config.dictConfig(
             {
                 "version": 1,
@@ -71,7 +73,7 @@ def setup_logging(**kwargs):
                 },
             }
         )
-    elif verbosity == 3:
+    elif verbose == 3:
         logging.config.dictConfig(
             {
                 "version": 1,
@@ -102,3 +104,5 @@ def setup_logging(**kwargs):
                 },
             }
         )
+    else:
+        raise ValueError("'verbose' must be one of: 0, 1, 2, 3")
diff --git a/setup.py b/setup.py
index e69423e8..1a8ebb36 100755
--- a/setup.py
+++ b/setup.py
@@ -51,7 +51,7 @@ setup(
         "owslib",
         "geojson",
         "pyproj",
-        "usgs",
+        "usgs >= 0.3.0",
         "boto3",
         "flasgger",
         "jsonpath-ng",
diff --git a/tox.ini b/tox.ini
index 9c417445..8e8362c5 100644
--- a/tox.ini
+++ b/tox.ini
@@ -36,6 +36,9 @@ commands =
         --with-doctest\
         --cover-erase\
         --cover-inclusive \
+        # To fix an issue on Windows from Python 3.8 that would break
+        # tests discovery.
+        --traverse-namespace \
         {posargs:--ignore-files=test_end_to_end.py {toxinidir}/eodag {toxinidir}/tests}
 deps =
     -r{toxinidir}/requirements-dev.txt


Test patch:
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index a8255fa1..e5b0d2e8 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -29,7 +29,7 @@ jobs:
     strategy:
       matrix:
         python-version: [3.6, 3.9]
-        os: [ubuntu-latest]
+        os: [ubuntu-latest, windows-latest]
     steps:
     - name: Checkout the repo
       uses: actions/checkout@v2
diff --git a/tests/context.py b/tests/context.py
index c0f3a8b0..83d9354e 100644
--- a/tests/context.py
+++ b/tests/context.py
@@ -25,13 +25,13 @@ import sys
 sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..")))
 
 from eodag import EODataAccessGateway, api, config
-from eodag.api.core import DEFAULT_ITEMS_PER_PAGE
+from eodag.api.core import DEFAULT_ITEMS_PER_PAGE, DEFAULT_MAX_ITEMS_PER_PAGE
 from eodag.api.product import EOProduct
 from eodag.api.product.drivers import DRIVERS
 from eodag.api.product.drivers.base import NoDriver
 from eodag.api.search_result import SearchResult
 from eodag.cli import download, eodag, list_pt, search_crunch
-from eodag.config import load_default_config
+from eodag.config import load_default_config, merge_configs
 from eodag.plugins.authentication.base import Authentication
 from eodag.plugins.crunch.filter_date import FilterDate
 from eodag.plugins.crunch.filter_latest_tpl_name import FilterLatestByName
@@ -49,6 +49,8 @@ from eodag.utils.exceptions import (
     AuthenticationError,
     DownloadError,
     MisconfiguredError,
+    NoMatchingProductType,
+    PluginImplementationError,
     UnsupportedDatasetAddressScheme,
     UnsupportedProvider,
     ValidationError,
diff --git a/tests/resources/fake_ext_plugin/eodag_fakeplugin/__init__.py b/tests/resources/fake_ext_plugin/eodag_fakeplugin/__init__.py
new file mode 100644
index 00000000..ec2fd1e3
--- /dev/null
+++ b/tests/resources/fake_ext_plugin/eodag_fakeplugin/__init__.py
@@ -0,0 +1,28 @@
+# -*- coding: utf-8 -*-
+# Copyright 2021, CS GROUP - France, http://www.c-s.fr
+#
+# This file is part of EODAG project
+#     https://www.github.com/CS-SI/EODAG
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# Filter Cython warnings
+# Apparently (see
+# https://stackoverflow.com/questions/40845304/runtimewarning-numpy-dtype-size-changed-may-indicate-binary-incompatibility     # noqa
+# and https://github.com/numpy/numpy/issues/11788) this is caused by Cython and affect pre-built Python packages that
+# depends on numpy and ship with a pre-built version of numpy that is older than 1.15.1 (where the warning is silenced
+# exactly as below)
+"""EODAG fake_ext_plugin."""
+
+
+class FakePluginAPI:
+    pass
diff --git a/tests/resources/fake_ext_plugin/providers.yml b/tests/resources/fake_ext_plugin/eodag_fakeplugin/providers.yml
similarity index 100%
rename from tests/resources/fake_ext_plugin/providers.yml
rename to tests/resources/fake_ext_plugin/eodag_fakeplugin/providers.yml
diff --git a/tests/test_cli.py b/tests/test_cli.py
index c3c7d742..94e07cc3 100644
--- a/tests/test_cli.py
+++ b/tests/test_cli.py
@@ -29,6 +29,7 @@ from pkg_resources import resource_filename
 from eodag.utils import GENERIC_PRODUCT_TYPE
 from tests import TEST_RESOURCES_PATH
 from tests.context import (
+    DEFAULT_ITEMS_PER_PAGE,
     AuthenticationError,
     MisconfiguredError,
     download,
@@ -153,7 +154,7 @@ class TestEodagCli(unittest.TestCase):
             )
             api_obj = dag.return_value
             api_obj.search.assert_called_once_with(
-                items_per_page=20,
+                items_per_page=DEFAULT_ITEMS_PER_PAGE,
                 page=1,
                 startTimeFromAscendingNode=None,
                 completionTimeFromAscendingNode=None,
@@ -198,7 +199,7 @@ class TestEodagCli(unittest.TestCase):
             )
             api_obj = dag.return_value
             api_obj.search.assert_called_once_with(
-                items_per_page=20,
+                items_per_page=DEFAULT_ITEMS_PER_PAGE,
                 page=1,
                 startTimeFromAscendingNode=None,
                 completionTimeFromAscendingNode=None,
@@ -295,7 +296,7 @@ class TestEodagCli(unittest.TestCase):
                 user_conf_file_path=conf_file, locations_conf_path=None
             )
             api_obj.search.assert_called_once_with(
-                items_per_page=20, page=1, **criteria
+                items_per_page=DEFAULT_ITEMS_PER_PAGE, page=1, **criteria
             )
             api_obj.crunch.assert_called_once_with(
                 search_results, search_criteria=criteria, **{cruncher: {}}
@@ -328,6 +329,41 @@ class TestEodagCli(unittest.TestCase):
                 **{cruncher: {"minimum_overlap": 10}}
             )
 
+    @mock.patch("eodag.cli.EODataAccessGateway", autospec=True)
+    def test_eodag_search_all(self, dag):
+        """Calling eodag search with --bbox argument valid"""
+        with self.user_conf() as conf_file:
+            product_type = "whatever"
+            self.runner.invoke(
+                eodag,
+                [
+                    "search",
+                    "--conf",
+                    conf_file,
+                    "-p",
+                    product_type,
+                    "-g",
+                    "POLYGON ((1 43, 1 44, 2 44, 2 43, 1 43))",
+                    "--all",
+                ],
+            )
+            api_obj = dag.return_value
+            api_obj.search_all.assert_called_once_with(
+                items_per_page=None,
+                startTimeFromAscendingNode=None,
+                completionTimeFromAscendingNode=None,
+                cloudCover=None,
+                geometry="POLYGON ((1 43, 1 44, 2 44, 2 43, 1 43))",
+                instrument=None,
+                platform=None,
+                platformSerialIdentifier=None,
+                processingLevel=None,
+                sensorType=None,
+                productType=product_type,
+                id=None,
+                locations=None,
+            )
+
     def test_eodag_list_product_type_ok(self):
         """Calling eodag list without provider return all supported product types"""
         all_supported_product_types = [
diff --git a/tests/test_config.py b/tests/test_config.py
index 2d01ad25..936df2b2 100644
--- a/tests/test_config.py
+++ b/tests/test_config.py
@@ -23,7 +23,7 @@ from io import StringIO
 
 import yaml.parser
 
-from tests.context import ValidationError, config
+from tests.context import ValidationError, config, merge_configs
 
 
 class TestProviderConfig(unittest.TestCase):
@@ -136,6 +136,48 @@ class TestProviderConfig(unittest.TestCase):
         self.assertTrue(hasattr(provider_config.api, "newParam"))
         self.assertEqual(provider_config.api.newParam, "val")
 
+    def test_provider_config_merge(self):
+        """Merge 2 providers configs"""
+        config_stream1 = StringIO(
+            """!provider
+                name: provider1
+                provider_param: val
+                provider_param2: val2
+                api: !plugin
+                    type: MyPluginClass
+                    plugin_param1: value1
+                    pluginParam2: value2
+        """
+        )
+        config_stream2 = StringIO(
+            """!provider
+                name: provider1
+                provider_param: val1
+                provider_param3: val3
+                api: !plugin
+                    type: MyPluginClass
+                    pluginParam2: value3
+        """
+        )
+        provider_config1 = yaml.load(config_stream1, Loader=yaml.Loader)
+        provider_config2 = yaml.load(config_stream2, Loader=yaml.Loader)
+
+        providers_config = {
+            "provider1": provider_config1,
+            "provider2": provider_config1,
+        }
+
+        merge_configs(
+            providers_config,
+            {"provider1": provider_config2, "provider3": provider_config1},
+        )
+        self.assertEqual(len(providers_config), 3)
+        self.assertEqual(providers_config["provider1"].provider_param, "val1")
+        self.assertEqual(providers_config["provider1"].provider_param2, "val2")
+        self.assertEqual(providers_config["provider1"].provider_param3, "val3")
+        self.assertEqual(providers_config["provider1"].api.plugin_param1, "value1")
+        self.assertEqual(providers_config["provider1"].api.pluginParam2, "value3")
+
 
 class TestPluginConfig(unittest.TestCase):
     def test_plugin_config_valid(self):
@@ -314,7 +356,7 @@ class TestConfigFunctions(unittest.TestCase):
         self.assertEqual(peps_conf.download.outputs_prefix, "/data")
 
         theia_conf = default_config["theia"]
-        self.assertEqual(theia_conf.download.outputs_prefix[-4:], "/tmp")
+        self.assertEqual(theia_conf.download.outputs_prefix, tempfile.gettempdir())
 
         my_new_provider_conf = default_config["my_new_provider"]
         self.assertEqual(my_new_provider_conf.priority, 4)
diff --git a/tests/test_end_to_end.py b/tests/test_end_to_end.py
index 7f387df8..294d0d5e 100644
--- a/tests/test_end_to_end.py
+++ b/tests/test_end_to_end.py
@@ -38,8 +38,8 @@ THEIA_SEARCH_ARGS = [
 SOBLOO_SEARCH_ARGS = [
     "sobloo",
     "S2_MSI_L1C",
-    "2020-05-01",
-    "2020-06-01",
+    "2021-01-01",
+    "2021-02-01",
     [0.2563590566012408, 43.19555008715042, 2.379835675499976, 43.907759172380565],
 ]
 PEPS_BEFORE_20161205_SEARCH_ARGS = [
@@ -140,17 +140,17 @@ class EndToEndBase(unittest.TestCase):
         - Return one product to be downloaded
         """
         search_criteria = {
+            "productType": product_type,
             "startTimeFromAscendingNode": start,
             "completionTimeFromAscendingNode": end,
             "geom": geom,
         }
+        if items_per_page:
+            search_criteria["items_per_page"] = items_per_page
+        if page:
+            search_criteria["page"] = page
         self.eodag.set_preferred_provider(provider)
-        results, nb_results = self.eodag.search(
-            productType=product_type,
-            page=page,
-            items_per_page=items_per_page,
-            **search_criteria
-        )
+        results, nb_results = self.eodag.search(**search_criteria)
         if offline:
             results = [
                 prod
@@ -165,6 +165,38 @@ class EndToEndBase(unittest.TestCase):
         else:
             return results
 
+    def execute_search_all(
+        self,
+        provider,
+        product_type,
+        start,
+        end,
+        geom,
+        items_per_page=None,
+        check_products=True,
+    ):
+        """Search all the products on provider:
+
+        - First set the preferred provider as the one given in parameter
+        - Then do the search_all
+        - Then ensure that at least the first result originates from the provider
+        - Return all the products
+        """
+        search_criteria = {
+            "startTimeFromAscendingNode": start,
+            "completionTimeFromAscendingNode": end,
+            "geom": geom,
+        }
+        self.eodag.set_preferred_provider(provider)
+        results = self.eodag.search_all(
+            productType=product_type, items_per_page=items_per_page, **search_criteria
+        )
+        if check_products:
+            self.assertGreater(len(results), 0)
+            one_product = results[0]
+            self.assertEqual(one_product.provider, provider)
+        return results
+
 
 # @unittest.skip("skip auto run")
 class TestEODagEndToEnd(EndToEndBase):
@@ -269,17 +301,17 @@ class TestEODagEndToEnd(EndToEndBase):
 
     def test_end_to_end_search_download_usgs(self):
         product = self.execute_search(*USGS_SEARCH_ARGS)
-        expected_filename = "{}.tar.bz".format(product.properties["title"])
+        expected_filename = "{}.tar.gz".format(product.properties["title"])
         self.execute_download(product, expected_filename)
 
-    def test_end_to_end_search_download_airbus(self):
+    def test_end_to_end_search_download_sobloo(self):
         product = self.execute_search(*SOBLOO_SEARCH_ARGS)
         expected_filename = "{}.zip".format(product.properties["title"])
         self.execute_download(product, expected_filename)
 
-    def test_end_to_end_search_download_airbus_noresult(self):
+    def test_end_to_end_search_download_sobloo_noresult(self):
         """Requesting a page on sobloo with no results must return an empty SearchResult"""
-        # As of 2021-02-23 this search at page 1 returns 68 products, so at page 2 there
+        # As of 2021-03-19 this search at page 1 returns 66 products, so at page 2 there
         # are no products available and sobloo returns a response without products (`hits`).
         product = self.execute_search(
             *SOBLOO_SEARCH_ARGS, page=2, items_per_page=100, check_product=False
@@ -357,7 +389,34 @@ class TestEODagEndToEnd(EndToEndBase):
         )
         self.assertGreaterEqual(os.stat(quicklook_file_path).st_size, 2 ** 5)
 
+    def test__search_by_id_sobloo(self):
+        # A single test with sobloo to check that _search_by_id returns
+        # correctly the exact product looked for.
+        uid = "S2A_MSIL1C_20200810T030551_N0209_R075_T53WPU_20200810T050611"
+        provider = "sobloo"
 
+        products, _ = self.eodag._search_by_id(uid=uid, provider=provider)
+        product = products[0]
+
+        self.assertEqual(product.properties["id"], uid)
+
+    def test_end_to_end_search_all_mundi_default(self):
+        # 23/03/2021: Got 16 products for this search
+        results = self.execute_search_all(*MUNDI_SEARCH_ARGS)
+        self.assertGreater(len(results), 10)
+
+    def test_end_to_end_search_all_mundi_iterate(self):
+        # 23/03/2021: Got 16 products for this search
+        results = self.execute_search_all(*MUNDI_SEARCH_ARGS, items_per_page=10)
+        self.assertGreater(len(results), 10)
+
+    def test_end_to_end_search_all_astraea_eod_iterate(self):
+        # 23/03/2021: Got 39 products for this search
+        results = self.execute_search_all(*ASTRAE_EOD_SEARCH_ARGS, items_per_page=10)
+        self.assertGreater(len(results), 10)
+
+
+# @unittest.skip("skip auto run")
 class TestEODagEndToEndWrongCredentials(EndToEndBase):
     """Make real case tests with wrong credentials. This assumes the existence of a
     wrong_credentials_cong.yml file in resources folder named user_conf.yml"""
diff --git a/tests/units/test_core.py b/tests/units/test_core.py
index df78fce7..83b4f975 100644
--- a/tests/units/test_core.py
+++ b/tests/units/test_core.py
@@ -17,17 +17,24 @@
 # limitations under the License.
 
 import glob
+import json
 import os
 import shutil
 import unittest
+from copy import deepcopy
 
 from shapely import wkt
-from shapely.geometry import LineString, MultiPolygon
+from shapely.geometry import LineString, MultiPolygon, Polygon
 
 from eodag.utils import GENERIC_PRODUCT_TYPE
 from tests import TEST_RESOURCES_PATH
 from tests.context import (
+    DEFAULT_MAX_ITEMS_PER_PAGE,
     EODataAccessGateway,
+    EOProduct,
+    NoMatchingProductType,
+    PluginImplementationError,
+    SearchResult,
     UnsupportedProvider,
     get_geometry_from_various,
     makedirs,
@@ -122,6 +129,7 @@ class TestCore(unittest.TestCase):
             "sobloo",
             "onda",
             "mundi",
+            "usgs",
             "creodias",
             "astraea_eod",
             "usgs_satapi_aws",
@@ -143,25 +151,10 @@ class TestCore(unittest.TestCase):
         "earth_search",
     ]
 
-    def setUp(self):
-        super(TestCore, self).setUp()
-        self.dag = EODataAccessGateway()
-
-    def tearDown(self):
-        super(TestCore, self).tearDown()
-        for old in glob.glob1(self.dag.conf_dir, "*.old") + glob.glob1(
-            self.dag.conf_dir, ".*.old"
-        ):
-            old_path = os.path.join(self.dag.conf_dir, old)
-            if os.path.exists(old_path):
-                try:
-                    os.remove(old_path)
-                except OSError:
-                    shutil.rmtree(old_path)
-        if os.getenv("EODAG_CFG_FILE") is not None:
-            os.environ.pop("EODAG_CFG_FILE")
-        if os.getenv("EODAG_LOCS_CFG_FILE") is not None:
-            os.environ.pop("EODAG_LOCS_CFG_FILE")
+    @classmethod
+    def setUpClass(cls):
+        cls.dag = EODataAccessGateway()
+        cls.conf_dir = os.path.join(os.path.expanduser("~"), ".config", "eodag")
 
     def test_supported_providers_in_unit_test(self):
         """Every provider must be referenced in the core unittest SUPPORTED_PROVIDERS class attribute"""  # noqa
@@ -216,25 +209,75 @@ class TestCore(unittest.TestCase):
         self.assertIn("sensorType", structure)
         self.assertIn(structure["ID"], self.SUPPORTED_PRODUCT_TYPES)
 
-    def test_core_object_creates_config_standard_location(self):
-        """The core object must create a user config file in standard user config location on instantiation"""  # noqa
-        self.execution_involving_conf_dir(inspect="eodag.yml")
-
-    def test_core_object_creates_index_if_not_exist(self):
-        """The core object must create an index in user config directory"""
-        self.execution_involving_conf_dir(inspect=".index")
-
     @mock.patch("eodag.api.core.open_dir", autospec=True)
     @mock.patch("eodag.api.core.exists_in", autospec=True, return_value=True)
     def test_core_object_open_index_if_exists(self, exists_in_mock, open_dir_mock):
         """The core object must use the existing index dir if any"""
-        conf_dir = os.path.join(os.path.expanduser("~"), ".config", "eodag")
-        index_dir = os.path.join(conf_dir, ".index")
+        index_dir = os.path.join(self.conf_dir, ".index")
         if not os.path.exists(index_dir):
             makedirs(index_dir)
         EODataAccessGateway()
         open_dir_mock.assert_called_with(index_dir)
 
+    def test_core_object_set_default_locations_config(self):
+        """The core object must set the default locations config on instantiation"""  # noqa
+        default_shpfile = os.path.join(
+            self.conf_dir, "shp", "ne_110m_admin_0_map_units.shp"
+        )
+        self.assertIsInstance(self.dag.locations_config, list)
+        self.assertEqual(
+            self.dag.locations_config,
+            [dict(attr="ADM0_A3_US", name="country", path=default_shpfile)],
+        )
+
+    def test_core_object_locations_file_not_found(self):
+        """The core object must set the locations to an empty list when the file is not found"""  # noqa
+        dag = EODataAccessGateway(locations_conf_path="no_locations.yml")
+        self.assertEqual(dag.locations_config, [])
+
+    def test_rebuild_index(self):
+        """Change eodag version and check that whoosh index is rebuilt"""
+        index_dir = os.path.join(self.dag.conf_dir, ".index")
+        index_dir_mtime = os.path.getmtime(index_dir)
+
+        self.assertNotEqual(self.dag.get_version(), "fake-version")
+
+        with mock.patch(
+            "eodag.api.core.EODataAccessGateway.get_version",
+            autospec=True,
+            return_value="fake-version",
+        ):
+            self.assertEqual(self.dag.get_version(), "fake-version")
+
+            self.dag.build_index()
+
+            # check that index_dir has beeh re-created
+            self.assertNotEqual(os.path.getmtime(index_dir), index_dir_mtime)
+
+
+class TestCoreConfWithEnvVar(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):
+        cls.dag = EODataAccessGateway()
+
+    @classmethod
+    def tearDownClass(cls):
+        if os.getenv("EODAG_CFG_FILE") is not None:
+            os.environ.pop("EODAG_CFG_FILE")
+        if os.getenv("EODAG_LOCS_CFG_FILE") is not None:
+            os.environ.pop("EODAG_LOCS_CFG_FILE")
+
+    def test_core_object_prioritize_locations_file_in_envvar(self):
+        """The core object must use the locations file pointed to by the EODAG_LOCS_CFG_FILE env var"""  # noqa
+        os.environ["EODAG_LOCS_CFG_FILE"] = os.path.join(
+            TEST_RESOURCES_PATH, "file_locations_override.yml"
+        )
+        dag = EODataAccessGateway()
+        self.assertEqual(
+            dag.locations_config,
+            [dict(attr="dummyattr", name="dummyname", path="dummypath.shp")],
+        )
+
     def test_core_object_prioritize_config_file_in_envvar(self):
         """The core object must use the config file pointed to by the EODAG_CFG_FILE env var"""  # noqa
         os.environ["EODAG_CFG_FILE"] = os.path.join(
@@ -246,6 +289,24 @@ class TestCore(unittest.TestCase):
         # peps outputs prefix is set to /data
         self.assertEqual(dag.providers_config["peps"].download.outputs_prefix, "/data")
 
+
+class TestCoreInvolvingConfDir(unittest.TestCase):
+    def setUp(self):
+        super(TestCoreInvolvingConfDir, self).setUp()
+        self.dag = EODataAccessGateway()
+
+    def tearDown(self):
+        super(TestCoreInvolvingConfDir, self).tearDown()
+        for old in glob.glob1(self.dag.conf_dir, "*.old") + glob.glob1(
+            self.dag.conf_dir, ".*.old"
+        ):
+            old_path = os.path.join(self.dag.conf_dir, old)
+            if os.path.exists(old_path):
+                try:
+                    os.remove(old_path)
+                except OSError:
+                    shutil.rmtree(old_path)
+
     def execution_involving_conf_dir(self, inspect=None):
         """Check that the path(s) inspected (str, list) are created after the instantation
         of EODataAccessGateway. If they were already there, rename them (.old), instantiate,
@@ -273,36 +334,23 @@ class TestCore(unittest.TestCase):
                         os.unlink(current)
                     shutil.move(old, current)
 
+    def test_core_object_creates_config_standard_location(self):
+        """The core object must create a user config file in standard user config location on instantiation"""  # noqa
+        self.execution_involving_conf_dir(inspect="eodag.yml")
+
+    def test_core_object_creates_index_if_not_exist(self):
+        """The core object must create an index in user config directory"""
+        self.execution_involving_conf_dir(inspect=".index")
+
     def test_core_object_creates_locations_standard_location(self):
         """The core object must create a locations config file and a shp dir in standard user config location on instantiation"""  # noqa
         self.execution_involving_conf_dir(inspect=["locations.yml", "shp"])
 
-    def test_core_object_set_default_locations_config(self):
-        """The core object must set the default locations config on instantiation"""  # noqa
-        dag = EODataAccessGateway()
-        conf_dir = os.path.join(os.path.expanduser("~"), ".config", "eodag")
-        default_shpfile = os.path.join(conf_dir, "shp", "ne_110m_admin_0_map_units.shp")
-        self.assertIsInstance(dag.locations_config, list)
-        self.assertEqual(
-            dag.locations_config,
-            [dict(attr="ADM0_A3_US", name="country", path=default_shpfile)],
-        )
 
-    def test_core_object_locations_file_not_found(self):
-        """The core object must set the locations to an empty list when the file is not found"""  # noqa
-        dag = EODataAccessGateway(locations_conf_path="no_locations.yml")
-        self.assertEqual(dag.locations_config, [])
-
-    def test_core_object_prioritize_locations_file_in_envvar(self):
-        """The core object must use the locations file pointed to by the EODAG_LOCS_CFG_FILE env var"""  # noqa
-        os.environ["EODAG_LOCS_CFG_FILE"] = os.path.join(
-            TEST_RESOURCES_PATH, "file_locations_override.yml"
-        )
-        dag = EODataAccessGateway()
-        self.assertEqual(
-            dag.locations_config,
-            [dict(attr="dummyattr", name="dummyname", path="dummypath.shp")],
-        )
+class TestCoreGeometry(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):
+        cls.dag = EODataAccessGateway()
 
     def test_get_geometry_from_various_no_locations(self):
         """The search geometry can be set from a dict, list, tuple, WKT string or shapely geom"""
@@ -415,9 +463,8 @@ class TestCore(unittest.TestCase):
             locations_config, locations=dict(country="FRA"), geometry=geometry
         )
         self.assertIsInstance(geom_combined, MultiPolygon)
-        self.assertEquals(
-            len(geom_combined), 4
-        )  # France + Guyana + Corsica + somewhere over Poland
+        # France + Guyana + Corsica + somewhere over Poland
+        self.assertEquals(len(geom_combined), 4)
         geometry = {
             "lonmin": 0,
             "latmin": 50,
@@ -428,26 +475,604 @@ class TestCore(unittest.TestCase):
             locations_config, locations=dict(country="FRA"), geometry=geometry
         )
         self.assertIsInstance(geom_combined, MultiPolygon)
-        self.assertEquals(
-            len(geom_combined), 3
-        )  # The bounding box overlaps with France inland
+        # The bounding box overlaps with France inland
+        self.assertEquals(len(geom_combined), 3)
 
-    def test_rebuild_index(self):
-        """Change eodag version and check that whoosh index is rebuilt"""
 
-        index_dir = os.path.join(self.dag.conf_dir, ".index")
-        index_dir_mtime = os.path.getmtime(index_dir)
+class TestCoreSearch(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):
+        cls.dag = EODataAccessGateway()
+        # Get a SearchResult obj with 2 S2_MSI_L1C peps products
+        search_results_file = os.path.join(
+            TEST_RESOURCES_PATH, "eodag_search_result_peps.geojson"
+        )
+        with open(search_results_file, encoding="utf-8") as f:
+            search_results_geojson = json.load(f)
+        cls.search_results = SearchResult.from_geojson(search_results_geojson)
+        cls.search_results_size = len(cls.search_results)
+        # Change the id of these products, to emulate different products
+        search_results_data_2 = deepcopy(cls.search_results.data)
+        search_results_data_2[0].properties["id"] = "a"
+        search_results_data_2[1].properties["id"] = "b"
+        cls.search_results_2 = SearchResult(search_results_data_2)
+        cls.search_results_size_2 = len(cls.search_results_2)
 
-        self.assertNotEqual(self.dag.get_version(), "fake-version")
+    def test_guess_product_type_with_kwargs(self):
+        """guess_product_type must return the products matching the given kwargs"""
+        kwargs = dict(
+            instrument="MSI",
+            platform="SENTINEL2",
+            platformSerialIdentifier="S2A",
+        )
+        actual = self.dag.guess_product_type(**kwargs)
+        expected = [
+            "S2_MSI_L1C",
+            "S2_MSI_L2A",
+            "S2_MSI_L2A_MAJA",
+            "S2_MSI_L2B_MAJA_SNOW",
+            "S2_MSI_L2B_MAJA_WATER",
+            "S2_MSI_L3A_WASP",
+        ]
+        self.assertEqual(actual, expected)
 
-        with mock.patch(
-            "eodag.api.core.EODataAccessGateway.get_version",
-            autospec=True,
-            return_value="fake-version",
-        ):
-            self.assertEqual(self.dag.get_version(), "fake-version")
+    def test_guess_product_type_without_kwargs(self):
+        """guess_product_type must raise an exception when no kwargs are provided"""
+        with self.assertRaises(NoMatchingProductType):
+            self.dag.guess_product_type()
 
-            self.dag.build_index()
+    def test_guess_product_type_has_no_limit(self):
+        """guess_product_type must run a whoosh search without any limit"""
+        # Filter that should give more than 10 products referenced in the catalog.
+        opt_prods = [
+            p for p in self.dag.list_product_types() if p["sensorType"] == "OPTICAL"
+        ]
+        if len(opt_prods) <= 10:
+            self.skipTest("This test requires that more than 10 products are 'OPTICAL'")
+        guesses = self.dag.guess_product_type(
+            sensorType="OPTICAL",
+        )
+        self.assertGreater(len(guesses), 10)
 
-            # check that index_dir has beeh re-created
-            self.assertNotEqual(os.path.getmtime(index_dir), index_dir_mtime)
+    def test__prepare_search_no_parameters(self):
+        """_prepare_search must create some kwargs even when no parameter has been provided"""  # noqa
+        prepared_search = self.dag._prepare_search()
+        expected = {
+            "geometry": None,
+            "productType": None,
+        }
+        self.assertDictContainsSubset(expected, prepared_search)
+        expected = set(["geometry", "productType", "auth", "search_plugin"])
+        self.assertSetEqual(expected, set(prepared_search))
+
+    def test__prepare_search_dates(self):
+        """_prepare_search must handle start & end dates"""
+        base = {
+            "start": "2020-01-01",
+            "end": "2020-02-01",
+        }
+        prepared_search = self.dag._prepare_search(**base)
+        expected = {
+            "startTimeFromAscendingNode": base["start"],
+            "completionTimeFromAscendingNode": base["end"],
+        }
+        self.assertDictContainsSubset(expected, prepared_search)
+
+    def test__prepare_search_geom(self):
+        """_prepare_search must handle geom, box and bbox"""
+        # The default way to provide a geom is through the 'geom' argument.
+        base = {"geom": (0, 50, 2, 52)}
+        # "geom": "POLYGON ((1 43, 1 44, 2 44, 2 43, 1 43))"
+        prepared_search = self.dag._prepare_search(**base)
+        self.assertIn("geometry", prepared_search)
+        # 'box' and 'bbox' are supported for backwards compatibility,
+        # The priority is geom > bbox > box
+        base = {"box": (0, 50, 2, 52)}
+        prepared_search = self.dag._prepare_search(**base)
+        self.assertIn("geometry", prepared_search)
+        base = {"bbox": (0, 50, 2, 52)}
+        prepared_search = self.dag._prepare_search(**base)
+        self.assertIn("geometry", prepared_search)
+        base = {
+            "geom": "POLYGON ((1 43, 1 44, 2 44, 2 43, 1 43))",
+            "bbox": (0, 50, 2, 52),
+            "box": (0, 50, 1, 51),
+        }
+        prepared_search = self.dag._prepare_search(**base)
+        self.assertIn("geometry", prepared_search)
+        self.assertNotIn("bbox", prepared_search)
+        self.assertNotIn("bbox", prepared_search)
+        self.assertIsInstance(prepared_search["geometry"], Polygon)
+
+    def test__prepare_search_locations(self):
+        """_prepare_search must handle a location search"""
+        # When locations where introduced they could be passed
+        # as regular kwargs. The new and recommended way to provide
+        # them is through the 'locations' parameter.
+        base = {"locations": {"country": "FRA"}}
+        prepared_search = self.dag._prepare_search(**base)
+        self.assertIn("geometry", prepared_search)
+
+        # TODO: Remove this when support for locations kwarg is dropped.
+        base = {"country": "FRA"}
+        prepared_search = self.dag._prepare_search(**base)
+        self.assertIn("geometry", prepared_search)
+        self.assertNotIn("country", prepared_search)
+
+    def test__prepare_search_product_type_provided(self):
+        """_prepare_search must handle when a product type is given"""
+        base = {"productType": "S2_MSI_L1C"}
+        prepared_search = self.dag._prepare_search(**base)
+        expected = {"productType": base["productType"]}
+        self.assertDictContainsSubset(expected, prepared_search)
+
+    def test__prepare_search_product_type_guess_it(self):
+        """_prepare_search must guess a product type when required to"""
+        # Uses guess_product_type to find the product matching
+        # the best the given params.
+        base = dict(
+            instrument="MSI",
+            platform="SENTINEL2",
+            platformSerialIdentifier="S2A",
+        )
+        prepared_search = self.dag._prepare_search(**base)
+        expected = {"productType": "S2_MSI_L1C", **base}
+        self.assertDictContainsSubset(expected, prepared_search)
+
+    def test__prepare_search_with_id(self):
+        """_prepare_search must handle a search by id"""
+        base = {"id": "dummy-id", "provider": "sobloo"}
+        prepared_search = self.dag._prepare_search(**base)
+        expected = base
+        self.assertDictEqual(expected, prepared_search)
+
+    def test__prepare_search_preserve_additional_kwargs(self):
+        """_prepare_search must preserve additional kwargs"""
+        base = {
+            "productType": "S2_MSI_L1C",
+            "cloudCover": 10,
+        }
+        prepared_search = self.dag._prepare_search(**base)
+        expected = base
+        self.assertDictContainsSubset(expected, prepared_search)
+
+    def test__prepare_search_search_plugin_has_known_product_properties(self):
+        """_prepare_search must attach the product properties to the search plugin"""
+        prev_fav_provider = self.dag.get_preferred_provider()[0]
+        try:
+            self.dag.set_preferred_provider("peps")
+            base = {"productType": "S2_MSI_L1C"}
+            prepared_search = self.dag._prepare_search(**base)
+            # Just check that the title has been set correctly. There are more (e.g.
+            # abstract, platform, etc.) but this is sufficient to check that the
+            # product_type_config dict has been created and populated.
+            self.assertEqual(
+                prepared_search["search_plugin"].config.product_type_config["title"],
+                "SENTINEL2 Level-1C",
+            )
+        finally:
+            self.dag.set_preferred_provider(prev_fav_provider)
+
+    def test__prepare_search_search_plugin_has_generic_product_properties(self):
+        """_prepare_search must be able to attach the generic product properties to the search plugin"""  # noqa
+        prev_fav_provider = self.dag.get_preferred_provider()[0]
+        try:
+            self.dag.set_preferred_provider("peps")
+            base = {"productType": "product_unknown_to_eodag"}
+            prepared_search = self.dag._prepare_search(**base)
+            # product_type_config is still created if the product is not known to eodag
+            # however it contains no data.
+            self.assertIsNone(
+                prepared_search["search_plugin"].config.product_type_config["title"],
+            )
+        finally:
+            self.dag.set_preferred_provider(prev_fav_provider)
+
+    def test__prepare_search_peps_plugins_product_available(self):
+        """_prepare_search must return the search and auth plugins when productType is defined"""  # noqa
+        prev_fav_provider = self.dag.get_preferred_provider()[0]
+        try:
+            self.dag.set_preferred_provider("peps")
+            base = {"productType": "S2_MSI_L1C"}
+            prepared_search = self.dag._prepare_search(**base)
+            self.assertEqual(prepared_search["search_plugin"].provider, "peps")
+            self.assertEqual(prepared_search["auth"].provider, "peps")
+        finally:
+            self.dag.set_preferred_provider(prev_fav_provider)
+
+    def test__prepare_search_no_plugins_when_search_by_id(self):
+        """_prepare_search must not return the search and auth plugins for a search by id"""  # noqa
+        base = {"id": "some_id", "provider": "some_provider"}
+        prepared_search = self.dag._prepare_search(**base)
+        self.assertNotIn("search_plugin", prepared_search)
+        self.assertNotIn("auth", prepared_search)
+
+    def test__prepare_search_peps_plugins_product_not_available(self):
+        """_prepare_search can use another set of search and auth plugins than the ones of the preferred provider"""  # noqa
+        # Document a special behaviour whereby the search and auth plugins don't
+        # correspond to the preferred one. This occurs whenever the searched product
+        # isn't available for the preferred provider but is made available by  another
+        # one. In that case peps provides it and happens to be the first one on the list
+        # of providers that make it available.
+        prev_fav_provider = self.dag.get_preferred_provider()[0]
+        try:
+            self.dag.set_preferred_provider("theia")
+            base = {"productType": "S2_MSI_L1C"}
+            prepared_search = self.dag._prepare_search(**base)
+            self.assertEqual(prepared_search["search_plugin"].provider, "peps")
+            self.assertEqual(prepared_search["auth"].provider, "peps")
+        finally:
+            self.dag.set_preferred_provider(prev_fav_provider)
+
+    @mock.patch("eodag.plugins.search.qssearch.QueryStringSearch", autospec=True)
+    def test__do_search_counts_by_default(self, search_plugin):
+        """__do_search must create a count query by default"""
+        search_plugin.provider = "peps"
+        search_plugin.query.return_value = (
+            self.search_results.data,  # a list must be returned by .query
+            self.search_results_size,
+        )
+        sr, estimate = self.dag._do_search(search_plugin=search_plugin)
+        self.assertIsInstance(sr, SearchResult)
+        self.assertEqual(len(sr), self.search_results_size)
+        self.assertEqual(estimate, self.search_results_size)
+
+    @mock.patch("eodag.plugins.search.qssearch.QueryStringSearch", autospec=True)
+    def test__do_search_without_count(self, search_plugin):
+        """__do_search must be able to create a query without a count"""
+        search_plugin.provider = "peps"
+        search_plugin.query.return_value = (
+            self.search_results.data,
+            None,  # .query must return None if count is False
+        )
+        sr, estimate = self.dag._do_search(search_plugin=search_plugin, count=False)
+        self.assertIsNone(estimate)
+        self.assertEqual(len(sr), self.search_results_size)
+
+    @mock.patch("eodag.plugins.search.qssearch.QueryStringSearch", autospec=True)
+    def test__do_search_paginated_handle_no_count_returned(self, search_plugin):
+        """__do_search must provide a best estimate when a provider doesn't return a count"""  # noqa
+        search_plugin.provider = "peps"
+        # If the provider doesn't return a count, .query returns 0
+        search_plugin.query.return_value = (self.search_results.data, 0)
+        page = 4
+        sr, estimate = self.dag._do_search(
+            search_plugin=search_plugin,
+            page=page,
+            items_per_page=2,
+        )
+        self.assertEqual(len(sr), self.search_results_size)
+        # The count guess is: page * number_of_products_returned
+        self.assertEqual(estimate, page * self.search_results_size)
+
+    @mock.patch("eodag.plugins.search.qssearch.QueryStringSearch", autospec=True)
+    def test__do_search_paginated_handle_fuzzy_count(self, search_plugin):
+        """__do_search must provide a best estimate when a provider returns a fuzzy count"""  # noqa
+        search_plugin.provider = "peps"
+        search_plugin.query.return_value = (
+            self.search_results.data * 4,  # 8 products returned
+            22,  # fuzzy number, less than the real total count
+        )
+        page = 4
+        items_per_page = 10
+        sr, estimate = self.dag._do_search(
+            search_plugin=search_plugin,
+            page=page,
+            items_per_page=items_per_page,
+        )
+        # At page 4 with 10 items_per_page we should have a count of at least 30
+        # products available. However the provider returned 22. We know it's wrong.
+        # So we update the count with our current knowledge: 30 + 8
+        # Note that this estimate could still be largely inferior to the real total
+        # count.
+        expected_estimate = items_per_page * (page - 1) + len(sr)
+        self.assertEqual(len(sr), 8)
+        self.assertEqual(estimate, expected_estimate)
+
+    @mock.patch("eodag.plugins.search.qssearch.QueryStringSearch", autospec=True)
+    def test__do_search_paginated_handle_null_count(self, search_plugin):
+        """__do_search must provide a best estimate when a provider returns a null count"""  # noqa
+        # TODO: check the underlying implementation, it doesn't make so much sense since
+        # this case is already covered with nb_res = len(res) * page. This one uses
+        # nb_res = items_per_page * (page - 1) whick actually makes more sense. Choose
+        # one of them.
+        search_plugin.provider = "peps"
+        search_plugin.query.return_value = ([], 0)
+        page = 4
+        items_per_page = 10
+        sr, estimate = self.dag._do_search(
+            search_plugin=search_plugin,
+            page=page,
+            items_per_page=items_per_page,
+        )
+        expected_estimate = items_per_page * (page - 1)
+        self.assertEqual(len(sr), 0)
+        self.assertEqual(estimate, expected_estimate)
+
+    def test__do_search_does_not_raise_by_default(self):
+        """__do_search must not raise any error by default"""
+        # provider attribute required internally by __do_search for logging purposes.
+        class DummySearchPlugin:
+            provider = "peps"
+
+        sr, estimate = self.dag._do_search(search_plugin=DummySearchPlugin())
+        self.assertIsInstance(sr, SearchResult)
+        self.assertEqual(len(sr), 0)
+        self.assertEqual(estimate, 0)
+
+    def test__do_search_can_raise_errors(self):
+        """__do_search must not raise any error by default"""
+
+        class DummySearchPlugin:
+            provider = "peps"
+
+        # AttributeError raised when .query is tried to be accessed on the dummy plugin.
+        with self.assertRaises(AttributeError):
+            self.dag._do_search(search_plugin=DummySearchPlugin(), raise_errors=True)
+
+    @mock.patch("eodag.plugins.search.qssearch.QueryStringSearch", autospec=True)
+    def test__do_search_query_products_must_be_a_list(self, search_plugin):
+        """__do_search expects that each search plugin returns a list of products."""
+        search_plugin.provider = "peps"
+        search_plugin.query.return_value = (
+            self.search_results,
+            self.search_results_size,
+        )
+        with self.assertRaises(PluginImplementationError):
+            self.dag._do_search(search_plugin=search_plugin, raise_errors=True)
+
+    @mock.patch("eodag.plugins.search.qssearch.QueryStringSearch", autospec=True)
+    def test__do_search_register_downloader_if_search_intersection(self, search_plugin):
+        """__do_search must register each product's downloader if search_intersection is not None"""  # noqa
+        search_plugin.provider = "peps"
+        search_plugin.query.return_value = (
+            self.search_results.data,
+            self.search_results_size,
+        )
+        sr, _ = self.dag._do_search(search_plugin=search_plugin)
+        for product in sr:
+            self.assertIsNotNone(product.downloader)
+
+    @mock.patch("eodag.plugins.search.qssearch.QueryStringSearch", autospec=True)
+    def test__do_search_doest_not_register_downloader_if_no_search_intersection(
+        self, search_plugin
+    ):
+        """__do_search must not register downloaders if search_intersection is None"""
+
+        class DummyProduct:
+            seach_intersecion = None
+
+        search_plugin.provider = "peps"
+        search_plugin.query.return_value = ([DummyProduct(), DummyProduct()], 2)
+        sr, _ = self.dag._do_search(search_plugin=search_plugin)
+        for product in sr:
+            self.assertIsNone(product.downloader)
+
+    @mock.patch("eodag.api.core.EODataAccessGateway._prepare_search", autospec=True)
+    @mock.patch("eodag.plugins.search.qssearch.QueryStringSearch", autospec=True)
+    def test_search_iter_page_returns_iterator(self, search_plugin, prepare_seach):
+        """search_iter_page must return an iterator"""
+        search_plugin.provider = "peps"
+        search_plugin.query.side_effect = [
+            (self.search_results.data, None),
+            (self.search_results_2.data, None),
+        ]
+
+        class DummyConfig:
+            pagination = {}
+
+        search_plugin.config = DummyConfig()
+        prepare_seach.return_value = dict(search_plugin=search_plugin)
+        page_iterator = self.dag.search_iter_page(items_per_page=2)
+        first_result_page = next(page_iterator)
+        self.assertIsInstance(first_result_page, SearchResult)
+        self.assertEqual(len(first_result_page), self.search_results_size)
+        second_result_page = next(page_iterator)
+        self.assertIsInstance(second_result_page, SearchResult)
+        self.assertEqual(len(second_result_page), self.search_results_size_2)
+
+    @mock.patch("eodag.api.core.EODataAccessGateway._prepare_search", autospec=True)
+    @mock.patch("eodag.plugins.search.qssearch.QueryStringSearch", autospec=True)
+    def test_search_iter_page_exhaust_get_all_pages_and_quit_early(
+        self, search_plugin, prepare_seach
+    ):
+        """search_iter_page must stop as soon as less than items_per_page products were retrieved"""  # noqa
+        search_plugin.provider = "peps"
+        search_plugin.query.side_effect = [
+            (self.search_results.data, None),
+            ([self.search_results_2.data[0]], None),
+        ]
+
+        class DummyConfig:
+            pagination = {}
+
+        search_plugin.config = DummyConfig()
+        prepare_seach.return_value = dict(search_plugin=search_plugin)
+        page_iterator = self.dag.search_iter_page(items_per_page=2)
+        all_page_results = list(page_iterator)
+        self.assertEqual(len(all_page_results), 2)
+        self.assertIsInstance(all_page_results[0], SearchResult)
+
+    @mock.patch("eodag.api.core.EODataAccessGateway._prepare_search", autospec=True)
+    @mock.patch("eodag.plugins.search.qssearch.QueryStringSearch", autospec=True)
+    def test_search_iter_page_exhaust_get_all_pages_no_products_last_page(
+        self, search_plugin, prepare_seach
+    ):
+        """search_iter_page must stop if the page doesn't return any product"""
+        search_plugin.provider = "peps"
+        search_plugin.query.side_effect = [
+            (self.search_results.data, None),
+            (self.search_results_2.data, None),
+            ([], None),
+        ]
+
+        class DummyConfig:
+            pagination = {}
+
+        search_plugin.config = DummyConfig()
+        prepare_seach.return_value = dict(search_plugin=search_plugin)
+        page_iterator = self.dag.search_iter_page(items_per_page=2)
+        all_page_results = list(page_iterator)
+        self.assertEqual(len(all_page_results), 2)
+        self.assertIsInstance(all_page_results[0], SearchResult)
+
+    @mock.patch("eodag.api.core.EODataAccessGateway._prepare_search", autospec=True)
+    @mock.patch("eodag.plugins.search.qssearch.QueryStringSearch", autospec=True)
+    def test_search_iter_page_does_not_handle_query_errors(
+        self, search_plugin, prepare_seach
+    ):
+        """search_iter_page must propagate errors"""
+        search_plugin.provider = "peps"
+        search_plugin.query.side_effect = AttributeError()
+        prepare_seach.return_value = dict(search_plugin=search_plugin)
+        page_iterator = self.dag.search_iter_page()
+        with self.assertRaises(AttributeError):
+            next(page_iterator)
+
+    @mock.patch(
+        "eodag.plugins.search.qssearch.QueryStringSearch._request", autospec=True
+    )
+    @mock.patch(
+        "eodag.plugins.search.qssearch.QueryStringSearch.normalize_results",
+        autospec=True,
+    )
+    def test_search_iter_page_must_reset_next_attrs_if_next_mechanism(
+        self, normalize_results, _request
+    ):
+        """search_iter_page must reset the search plugin if the next mechanism is used"""
+        # More specifically: next_page_url must be None and
+        # config.pagination["next_page_url_tpl"] must be equal to its original value.
+        _request.return_value.json.return_value = {
+            "features": [],
+            "links": [{"rel": "next", "href": "url/to/next/page"}],
+        }
+
+        p1 = EOProduct("dummy", dict(geometry="POINT (0 0)", id="1"))
+        p1.search_intersection = None
+        p2 = EOProduct("dummy", dict(geometry="POINT (0 0)", id="2"))
+        p2.search_intersection = None
+        normalize_results.side_effect = [[p1], [p2]]
+        dag = EODataAccessGateway()
+        dummy_provider_config = """
+        dummy_provider:
+            search:
+                type: QueryStringSearch
+                api_endpoint: https://api.my_new_provider/search
+                pagination:
+                    next_page_url_tpl: 'dummy_next_page_url_tpl'
+                    next_page_url_key_path: '$.links[?(@.rel="next")].href'
+                metadata_mapping:
+                    dummy: 'dummy'
+            products:
+                S2_MSI_L1C:
+                    productType: '{productType}'
+        """
+        dag.update_providers_config(dummy_provider_config)
+        dag.set_preferred_provider("dummy_provider")
+
+        page_iterator = dag.search_iter_page(productType="S2_MSI_L1C")
+        next(page_iterator)
+        search_plugin = next(
+            dag._plugins_manager.get_search_plugins(product_type="S2_MSI_L1C")
+        )
+        self.assertIsNone(search_plugin.next_page_url)
+        self.assertEqual(
+            search_plugin.config.pagination["next_page_url_tpl"],
+            "dummy_next_page_url_tpl",
+        )
+
+    @mock.patch("eodag.api.core.EODataAccessGateway._prepare_search", autospec=True)
+    @mock.patch("eodag.plugins.search.qssearch.QueryStringSearch", autospec=True)
+    def test_search_all_must_collect_them_all(self, search_plugin, prepare_seach):
+        """search_all must return all the products available"""  # noqa
+        search_plugin.provider = "peps"
+        search_plugin.query.side_effect = [
+            (self.search_results.data, None),
+            ([self.search_results_2.data[0]], None),
+        ]
+
+        class DummyConfig:
+            pagination = {}
+
+        search_plugin.config = DummyConfig()
+
+        # Infinite generator her because passing directly the dict to
+        # prepare_search.return_value (or side_effect) didn't work. One function
+        # would do a dict.pop("search_plugin") that would remove the item from the
+        # mocked return value. Later calls would then break
+        def yield_search_plugin():
+            while True:
+                yield {"search_plugin": search_plugin}
+
+        prepare_seach.side_effect = yield_search_plugin()
+        all_results = self.dag.search_all(items_per_page=2)
+        self.assertIsInstance(all_results, SearchResult)
+        self.assertEqual(len(all_results), 3)
+
+    @mock.patch("eodag.api.core.EODataAccessGateway.search_iter_page", autospec=True)
+    def test_search_all_use_max_items_per_page(self, mocked_search_iter_page):
+        """search_all must use the configured parameter max_items_per_page if available"""  # noqa
+        dag = EODataAccessGateway()
+        dummy_provider_config = """
+        dummy_provider:
+            search:
+                type: QueryStringSearch
+                api_endpoint: https://api.my_new_provider/search
+                pagination:
+                    max_items_per_page: 2
+                metadata_mapping:
+                    dummy: 'dummy'
+            products:
+                S2_MSI_L1C:
+                    productType: '{productType}'
+        """
+        mocked_search_iter_page.return_value = (self.search_results for _ in range(1))
+        dag.update_providers_config(dummy_provider_config)
+        dag.set_preferred_provider("dummy_provider")
+        dag.search_all(productType="S2_MSI_L1C")
+        self.assertEqual(mocked_search_iter_page.call_args[1]["items_per_page"], 2)
+
+    @mock.patch("eodag.api.core.EODataAccessGateway.search_iter_page", autospec=True)
+    def test_search_all_use_default_value(self, mocked_search_iter_page):
+        """search_all must use the DEFAULT_MAX_ITEMS_PER_PAGE if the provider's one wasn't configured"""  # noqa
+        dag = EODataAccessGateway()
+        dummy_provider_config = """
+        dummy_provider:
+            search:
+                type: QueryStringSearch
+                api_endpoint: https://api.my_new_provider/search
+                metadata_mapping:
+                    dummy: 'dummy'
+            products:
+                S2_MSI_L1C:
+                    productType: '{productType}'
+        """
+        mocked_search_iter_page.return_value = (self.search_results for _ in range(1))
+        dag.update_providers_config(dummy_provider_config)
+        dag.set_preferred_provider("dummy_provider")
+        dag.search_all(productType="S2_MSI_L1C")
+        self.assertEqual(
+            mocked_search_iter_page.call_args[1]["items_per_page"],
+            DEFAULT_MAX_ITEMS_PER_PAGE,
+        )
+
+    @mock.patch("eodag.api.core.EODataAccessGateway.search_iter_page", autospec=True)
+    def test_search_all_user_items_per_page(self, mocked_search_iter_page):
+        """search_all must use the value of items_per_page provided by the user"""
+        dag = EODataAccessGateway()
+        dummy_provider_config = """
+        dummy_provider:
+            search:
+                type: QueryStringSearch
+                api_endpoint: https://api.my_new_provider/search
+                metadata_mapping:
+                    dummy: 'dummy'
+            products:
+                S2_MSI_L1C:
+                    productType: '{productType}'
+        """
+        mocked_search_iter_page.return_value = (self.search_results for _ in range(1))
+        dag.update_providers_config(dummy_provider_config)
+        dag.set_preferred_provider("dummy_provider")
+        dag.search_all(productType="S2_MSI_L1C", items_per_page=7)
+        self.assertEqual(mocked_search_iter_page.call_args[1]["items_per_page"], 7)