Repository: pydap/pydap (python)

Issue:
Query the CMR to find OPeNDAP urls (NASA specific)
The CMR is a great resource for finding opendap urls when searching using the collection concept ID or dataset DOI. `earthaccess` uses the CMR to even apply lat/lon filters, but it returns non-opendap urls.

### Minimal example
The following is an example workflow I use to search for opendap urls. It makes use of the request library, which is already a dependency of pydap.

Requirements: Concept ID of the collection, or the dataset DOI. 
#### Step 1: Use DOI to get concept collection ID (skip if already has).
```python
import requests

cmr_session = requests.Session()

# CMR API base url
cmrurl='https://cmr.earthdata.nasa.gov/search/'

# user supplies here the DOI for the dataset. For example:
doi = '10.5067/ECL5M-OTS44'
doisearch = cmrurl + 'collections.json?doi=' + doi

concept_id = cmr_session.get(doisearch).json()['feed']['entry'][0]['id']
```

The above code demonstrated a way to query the `concept_id` from the dataset `doi`.  Now, it is possible to query opendap urls. It is even possible to filter using time ranges

#### Step 2: Create a temporal filter. One could also create a spatial filter (lat/lon). I have only worked with time filters

```python
import datetime as dt

start_date =  dt.datetime(1992, 1, 1) # For example 
end_date = dt.datetime(2017, 12, 31) # For example
dt_format = '%Y-%m-%dT%H:%M:%SZ' # format requirement for datetime search
temporal_str = start_date.strftime(dt_format) + ',' + end_date.strftime(dt_format)
```
#### Step 3: Use concept_id and time-range to query all matching OPeNDAP URLs

Now, with the temporal range, one can query OPeNDAP urls with a helper function:

```python
def cmr_opendap_urls(concept_id, time_range, _session=None):
    """
    Queries NASA's `Common Metadata Repository` to identify all OPeNDAP URLS
    given collection concept ID and temporal time range.
    """
    cmr_url = 'https://cmr.earthdata.nasa.gov/search/granules'
    if not _session:
        _session = requests.Session() 
    cmr_response = _session.get(cmr_url, params={'concept_id': concept_id,'temporal': time_range,'page_size': 500}, headers={'Accept': 'application/json'})
    granules = cmr_response.json()['feed']['entry']
    granules_urls = []
    
    # Filter and only retain the OPeNDAP URLs
    for granule in granules:
        item = next((item['href'] for item in granule['links'] if "opendap" in item["href"]), None)
        if item != None:
            granules_urls.append(item)
    return granules_urls
    
```
and now use it as:
```python
# the following takes < 1 second
granules_urls = cmr_opendap_urls(concept_id, temporal_str)
```
granules_urls is a list with potentially 1000s of OPeNDAP URLs.

# 

I think this little helper function should be included in the documentation, and as a function within pydap.




Pull request:
Add function to query opendap urls from CMR
<!-- Thank you very much for your hard work and contribution! -->

The following Pull Request:

- [x] Closes #494 - Enables searching for OPeNDAP URLs via the CMR. Should enable filtering by time and space (bounding box).
- [x] Tests are added to demonstrate the new or fixed behavior, and all tests pass
on a local environment. I

**NOTE** ~It is possible that testing will be hard to accomplish - it may need auth credentials.~ Tests run, but they are a bit slow.

# Example
```python
from pydap.client import get_cmr_urls
collection_concept_id = "C1991543728-POCLOUD"
granule_urls = get_cmr_urls(ccid=collection_concept_id)
```
returns 312 (cloud) opendap urls in < 1 second.

## TODO before merging

Enable the following arguments:
- [x] Filter by bounding_box (including multiple ones). 
- [x] Filter by circle (including multiple ones)
- [x] Filter by line (including multiple ones)
- [x] Filter by polygon (including multiple ones)
- [x] Search by DOI (instead of collection concept id)
~OPTIONAL:  Enable filter by shapefile (e.g. Google Earth kml).~ It seems pretty straightforward - but the xml shapefile has altitude. Needs parsing to remove altitude. It does not even work directly on the Earthdata search




Patch:
diff --git a/src/pydap/client.py b/src/pydap/client.py
index 33afe33..72775b8 100644
--- a/src/pydap/client.py
+++ b/src/pydap/client.py
@@ -44,6 +44,7 @@ lazy mechanism for function call, supporting any function. Eg, to call the
 
 """
 
+import datetime as dt
 import os
 import re
 import warnings
@@ -52,6 +53,7 @@ from io import BytesIO, open
 from os.path import commonprefix
 from urllib.parse import parse_qs, unquote, urlencode
 
+import requests
 from requests.exceptions import (
     ConnectionError,
     SSLError,
@@ -664,6 +666,307 @@ def patch_session_for_shared_dap_cache(
     session._dap_cache_configs.append(new_config)
 
 
+def get_cmr_urls(
+    ccid=None,
+    doi=None,
+    time_range=None,
+    bounding_box: list | dict | None = None,
+    point: list | dict | None = None,
+    polygon: list | dict | None = None,
+    line: list | dict | None = None,
+    circle: list | dict | None = None,
+    session=None,
+    limit=50,
+):
+    """
+    Get the granule OPeNDAP URLs associated with a given concept collection ID (ccid) or
+    collection DOI (doi) from the CMR API. This functions allows you to filter
+    the search by time range and spatial shapes (bounding box, point, polygon, line,
+    circle)
+
+    `NOTE`: A query could consist of multiple spatial parameters of different types, two
+    bounding boxes and a polygon for example. If multiple spatial parameters are
+    present, all the parameters irrespective of their type are AND-ed in a query. So, if
+    a query contains two bounding boxes and a polygon for example, it will return only
+    those collections which intersect both the bounding boxes and the polygon.
+
+    `NOTE`: The spatial parameters are not mutually exclusive. You can use multiple
+    spatial parameters in a single query. For example, you can use a bounding box and a
+    polygon in the same query. When the spatial parameters are different types, theser
+    are AND-ed together, meaning the search will return results that intersect all
+    spatial parameters. For example, if you use a bounding box and a polygon in the same
+    query, the results will include only those collections that intersect both the
+    bounding box andf the polygon. When the spatial parameters are the same type, they
+    are AND-ed together by default. However, it is possible to make them OR-ed by adding
+    an extra key-value pair to the bounding box, point, polygon, line, or circle
+    parameters (dict). The key is "Union" and the value can be `True` of `False`.
+    If `True`, the spatial parameters are OR-ed together. For example, if you use two
+    bounding boxes in the same query, and set the "Union" key to `True`, the results
+    will include all collections that intersect either of the bounding boxes. If
+    `False` (default), the results will include only those collections that intersect
+    both bounding boxes. The same applies to the other spatial parameters.
+
+    Parameters
+    ----------
+        ccid : str
+            The collection concept ID to search for.
+        doi : str
+            The DOI of the collection to search for. This is an alternative to using
+            the ccid parameter.
+        time_range : list | None
+            The time range to filter by. The time range is a list of two elements,
+            each element a datetime.datetime object, of a string in the format
+            YYYY-MM-DDTHH:MM:SSZ.
+            Example1: ["2023-01-01T00:00:00Z", "2023-12-31T23:59:59Z"]
+            Example2: [datetime.datetime(2023, 1, 1), datetime.datetime(2023, 12, 31)]
+
+        bounding_box : list | dict | None
+            The bounding box to filter by, in the format [west, south, east, north].
+            Supports multiple bounding boxes.
+
+            Example1: bounding_box = [lon1, lat1, lon2, lat2]
+            Example2: bounding_box = {"box1": [lon1, lat1, lon2, lat2],
+                                      "box2": [lon3, lat3, lon4, lat4]}
+            Example3: bounding_box = {"box1": [lon1, lat1, lon2, lat2],
+                                      "box2": [lon3, lat3, lon4, lat4],
+                                      "Union": True}
+            The "Union" key is optional (if not present is set to `False`).
+
+        point: list | dict | None
+            Search using a pair of values representing the point coordinates as
+            parameters, in the format [longitude, latitude]. Supports multiple
+            points, as a dictionary.
+
+                Example1: point = [lon1, lat1]
+                Example2: point = {"point1": [lon1, lat1], "point2": [lon2, lat2]}
+                Example3: point = {"point1": [lon1, lat1], "point2": [lon2, lat2],
+                                   "Union": True}
+            The "Union" key is optional (if not present is set to `False`).
+
+        polygon: list | dict | None
+            The polygon to filter by. Polygon points are provided in counter-clockwise
+            order. The last point should match the first point to close the polygon.
+            The values are listed comma separated in longitude latitude order. Supports
+            multiple polygons, in that case polygon MUST be a dictionary.
+
+            Examples:
+                polygon = [lon1, lat1, lon2, lat2, lon3, lat3, lon1, lat1]
+                polygon = {"poly1": [lon1, lat1, lon2, lat2, ..., lon1, lat1],
+                           "poly2": [lon1, lat1, lon2, lat2, ..., lon1, lat1],
+                            "Union": True}
+            The "Union" key is optional (if not present is set to `False`).
+
+        line: list | dict | None
+            Lines are provided as a list of comma separated values representing
+            coordinates of points along the line. The coordinates are listed in the
+            format [lon1, lat1, lon2, lat2, ...]. Multiple lines can be provided, and in
+            that scenario it must be a dictionary.
+            Examples:
+                line = [lon1, lat1, lon2, lat2, ...]
+                line = {'line1': [lon11, lat11, lon12, lat12, ...],
+                        'line2': [lon21, lat21, lon22, lat22, ...]}
+
+        circle: list | dict | None
+            Circle defines a circle area on the earth with a center point and a radius.
+            The center parameters must be 3 comma-separated numbers: longitude of the
+            center point, latitude of the center point, radius of the circle in meters.
+            Multiple circles can be provided as nested lists.
+            Example:
+                circle = [lon, lat, radius]
+                circle = {"circle1": [lon1, lat1, radius1],
+                          "circle2": [lon2, lat2, radius2]}
+
+        session : requests.Session | None
+            A requests session object. If None, a new session is created.
+
+        limit : int
+            The maximum number of results to return. Default is 50.
+
+    Returns:
+    ---------
+        list
+            A list of granule OPeNDAP URLs.
+
+    See:
+        https://cmr.earthdata.nasa.gov/search/site/docs/search/api.html
+        https://cmr.earthdata.nasa.gov/search/site/docs/search/api.html#c-polygon
+        https://cmr.earthdata.nasa.gov/search/site/docs/search/api.html#c-bounding-box
+        https://cmr.earthdata.nasa.gov/search/site/docs/search/api.html#c-point
+        https://cmr.earthdata.nasa.gov/search/site/docs/search/api.html#c-line
+    """
+
+    if not ccid and not doi:
+        raise ValueError("Either ccid or doi must be provided.")
+
+    cmr_url = "https://cmr.earthdata.nasa.gov/search/granules"
+    headers = {
+        "Accept": "application/vnd.nasa.cmr.umm+json",
+    }
+    if session is None or not isinstance(session, (requests.Session, CachedSession)):
+        session = create_session()
+    params = {"page_size": limit}
+
+    if doi:
+        doisearch = "https://cmr.earthdata.nasa.gov/search/collections.json?doi=" + doi
+        ccid = session.get(doisearch).json()["feed"]["entry"][0]["id"]
+
+    params["concept_id"] = ccid
+
+    if time_range and isinstance(time_range, list):
+        if len(time_range) != 2:
+            warnings.warns("time_range must be a list of two elements.")
+            return None
+        if all(isinstance(x, dt.datetime) for x in time_range):
+            dt_format = "%Y-%m-%dT%H:%M:%SZ"
+            temporal_str = (
+                time_range[0].strftime(dt_format)
+                + ","
+                + time_range[1].strftime(dt_format)
+            )
+        else:
+            try:
+                dt.datetime.strptime(time_range[0], "%Y-%m-%dT%H:%M:%SZ")
+                dt.datetime.strptime(time_range[1], "%Y-%m-%dT%H:%M:%SZ")
+                temporal_str = time_range[0] + "," + time_range[1]
+            except ValueError:
+                warnings.warns(
+                    "time_range must be a list of two elements each a string in the"
+                    " format YYYY-MM-DDTHH:MM:SSZ."
+                )
+                return None
+        params["temporal"] = temporal_str
+    elif time_range and not isinstance(time_range, list):
+        warnings.warns(
+            "time_range must be a list of two elements or a string in the format"
+            " YYYY-MM-DDTHH:MM:SSZ."
+        )
+        return None
+    if bounding_box:
+        if isinstance(bounding_box, list):
+            cmr_url += "?bounding_box%5B%5D=" + "%2C".join(str(x) for x in bounding_box)
+        elif isinstance(bounding_box, dict):
+            ces = []
+            if "Union" in bounding_box:
+                extra = bounding_box.pop("Union", None)
+            else:
+                extra = False
+            for key, value in bounding_box.items():
+                if isinstance(value, list):
+                    ces.append(
+                        "bounding_box%5B%5D=" + "%2C".join(str(x) for x in value)
+                    )
+                else:
+                    raise ValueError(
+                        "bounding_box must be a list or a dictionary of lists."
+                    )
+            cmr_url += "?" + "&".join(ces)
+            if extra:
+                cmr_url += "&options%5Bbounding_box%5D%5Bor%5D=true"
+        else:
+            raise ValueError("`bounding_box` must be a list or a dictionary of lists.")
+    if polygon:
+        if isinstance(polygon, list):
+            cmr_url += "?polygon%5B%5D=" + "%2C".join(str(x) for x in polygon)
+        elif isinstance(polygon, dict):
+            ces = []
+            if "Union" in polygon:
+                extra = polygon.pop("Union", None)
+            else:
+                extra = False
+            for key, value in polygon.items():
+                if isinstance(value, list):
+                    ces.append("polygon%5B%5D=" + "%2C".join(str(x) for x in value))
+                else:
+                    raise ValueError("polygon must be a list or a dictionary of lists.")
+            cmr_url += "?" + "&".join(ces)
+            if extra:
+                cmr_url += "&options%5Bpolygon%5D%5Bor%5D=true"
+        else:
+            raise ValueError("`polygon` must be a list or a dictionary of lists.")
+    if line:
+        if isinstance(line, list):
+            cmr_url += "?line%5B%5D=" + "%2C".join(str(x) for x in line)
+        elif isinstance(line, dict):
+            ces = []
+            if "Union" in line:
+                extra = line.pop("Union", None)
+            else:
+                extra = False
+            for key, value in line.items():
+                if isinstance(value, list):
+                    ces.append("line%5B%5D=" + "%2C".join(str(x) for x in value))
+                else:
+                    raise ValueError("line must be a list or a dictionary of lists.")
+            cmr_url += "?" + "&".join(ces)
+            if extra:
+                cmr_url += "&options%5Bline%5D%5Bor%5D=true"
+        else:
+            raise ValueError("`line` must be a list or a dictionary of lists.")
+    if circle:
+        if isinstance(circle, list):
+            cmr_url += "?circle%5B%5D=" + "%2C".join(str(x) for x in circle)
+        elif isinstance(circle, dict):
+            ces = []
+            if "Union" in circle:
+                extra = circle.pop("Union", None)
+            else:
+                extra = False
+            for key, value in circle.items():
+                if isinstance(value, list):
+                    ces.append("circle%5B%5D=" + "%2C".join(str(x) for x in value))
+                else:
+                    raise ValueError("circle must be a list or a dictionary of lists.")
+            cmr_url += "?" + "&".join(ces)
+            if extra:
+                cmr_url += "&options%5Bcircle%5D%5Bor%5D=true"
+        else:
+            raise ValueError("`circle` must be a list or a dictionary of lists.")
+    if point:
+        if isinstance(point, list):
+            cmr_url += "?point%5B%5D=" + "%2C".join(str(x) for x in point)
+        elif isinstance(point, dict):
+            ces = []
+            if "Union" in point:
+                extra = point.pop("Union", None)
+            else:
+                extra = False
+            for key, value in point.items():
+                if isinstance(value, list):
+                    ces.append("point%5B%5D=" + "%2C".join(str(x) for x in value))
+                else:
+                    raise ValueError("point must be a list or a dictionary of lists.")
+            cmr_url += "?" + "&".join(ces)
+            if extra:
+                cmr_url += "&options%5Bpoint%5D%5Bor%5D=true"
+        else:
+            raise ValueError("`point` must be a list or a dictionary of lists.")
+
+    try:
+        r = session.get(cmr_url, params=params, headers=headers)
+        r.raise_for_status()
+    except requests.exceptions.RequestException as e:
+        print(f"Error - something went wrong: {e}")
+        return None
+
+    cmr_response = r.json()
+    items = [
+        cmr_response["items"][i]["umm"]["RelatedUrls"]
+        for i in range(len(cmr_response["items"]))
+    ]
+    granule_urls = list(
+        {
+            d["URL"]
+            for item in items
+            for d in item
+            if (
+                d.get("Description") == "OPeNDAP request URL"
+                or d.get("Subtype") == "OPENDAP DATA"
+            )
+        }
+    )
+    return granule_urls
+
+
 if __name__ == "__main__":
     fname = "/home/griessbaum/Dropbox/UCSB/pydap_cpt/pydap_notebooks/data/"
     ds = open_file(fname + "coads_climatology.nc.dmr")


Test patch:
diff --git a/src/pydap/tests/test_client.py b/src/pydap/tests/test_client.py
index 3bb3730..5082d85 100644
--- a/src/pydap/tests/test_client.py
+++ b/src/pydap/tests/test_client.py
@@ -1,5 +1,6 @@
 """Test the pydap client."""
 
+import datetime as dt
 import os
 
 import numpy as np
@@ -9,6 +10,7 @@ import requests
 from ..client import (
     compute_base_url_prefix,
     consolidate_metadata,
+    get_cmr_urls,
     open_dmr,
     open_dods_url,
     open_file,
@@ -16,6 +18,7 @@ from ..client import (
     patch_session_for_shared_dap_cache,
 )
 from ..handlers.lib import BaseHandler
+from ..lib import _quote
 from ..model import BaseType, DatasetType, GridType
 from ..net import create_session
 from .datasets import SimpleGrid, SimpleSequence, SimpleStructure
@@ -692,3 +695,214 @@ def test_patch_session_for_shared_dap_cache(urls, cached_session):
 
     # assert that there is no new cached key
     assert len(cached_session.cache.urls()) == len(dimensions)
+
+
+ccid = "concept_id=C2076114664-LPCLOUD"
+start_date = dt.datetime(2020, 1, 1)
+end_date = dt.datetime(2020, 1, 31)
+dt_format = "%Y-%m-%dT%H:%M:%SZ"
+bbox1 = "bounding_box%5B%5D=-10%2C-5%2C10%2C5"
+bbox2 = "bounding_box%5B%5D=-11%2C-6%2C11%2C6"
+
+
+@pytest.mark.parametrize(
+    "param, expected",
+    [
+        [{"doi": "10.5067/ECL5M-OTS44"}, "concept_id=C1991543728-POCLOUD&page_size=50"],
+        [
+            {
+                "ccid": ccid.split("=")[-1],
+                "time_range": [start_date, end_date],
+            },
+            ccid + "&temporal=2020-01-01T00%3A00%3A00Z%2C2020-01-31T00%3A00%3A00Z",
+        ],
+        [
+            {
+                "ccid": ccid.split("=")[-1],
+                "time_range": [
+                    start_date.strftime(dt_format),
+                    end_date.strftime(dt_format),
+                ],
+            },
+            ccid + "&temporal=2020-01-01T00%3A00%3A00Z%2C2020-01-31T00%3A00%3A00Z",
+        ],
+        [
+            {
+                "ccid": ccid.split("=")[-1],
+                "bounding_box": list((-130.8, 41, -124, 45)),
+            },
+            ccid + "&bounding_box%5B%5D=-130.8%2C41%2C-124%2C45",
+        ],
+        [
+            {
+                "ccid": ccid.split("=")[-1],
+                "bounding_box": {
+                    "key1": list((-10, -5, 10, 5)),
+                    "key2": list((-11, -6, 11, 6)),
+                },
+            },
+            ccid + "&" + bbox1 + "&" + bbox2,
+        ],
+        [
+            {
+                "ccid": ccid.split("=")[-1],
+                "bounding_box": {
+                    "key1": list((-10, -5, 10, 5)),
+                    "key2": list((-11, -6, 11, 6)),
+                    "Union": True,
+                },
+            },
+            ccid
+            + "&"
+            + bbox1
+            + "&"
+            + bbox2
+            + "&options%5Bbounding_box%5D%5Bor%5D=true",
+        ],
+        [
+            {
+                "ccid": ccid.split("=")[-1],
+                "point": [100, 20],
+            },
+            ccid + "&point%5B%5D=100%2C20",
+        ],
+        [
+            {
+                "ccid": "C1991543728-POCLOUD",
+                "point": {"point1": [100, 20], "point2": [80, 20]},
+            },
+            "concept_id=C1991543728-POCLOUD"
+            + "&point%5B%5D=100%2C20&point%5B%5D=80%2C20",
+        ],
+        [
+            {
+                "ccid": "C1991543728-POCLOUD",
+                "point": {"point1": [100, 20], "point2": [80, 20], "Union": True},
+            },
+            "concept_id=C1991543728-POCLOUD"
+            + "&point%5B%5D=100%2C20&point%5B%5D=80%2C20"
+            + "&options%5Bpoint%5D%5Bor%5D=true",
+        ],
+        [
+            {
+                "ccid": "C1991543728-POCLOUD",
+                "polygon": [10, 10, 30, 10, 30, 20, 10, 20, 10, 10],
+            },
+            "concept_id=C1991543728-POCLOUD"
+            + "&polygon%5B%5D=10%2C10%2C30%2C10%2C30%2C20%2C10%2C20%2C10%2C10",
+        ],
+        [
+            {
+                "ccid": "C1991543728-POCLOUD",
+                "polygon": {
+                    "p1": [10, 10, 30, 10, 30, 20, 10, 20, 10, 10],
+                    "p2": [11, 11, 31, 11, 31, 21, 11, 21, 11, 11],
+                },
+            },
+            "concept_id=C1991543728-POCLOUD"
+            + "&polygon%5B%5D=10%2C10%2C30%2C10%2C30%2C20%2C10%2C20%2C10%2C10"
+            + "&polygon%5B%5D=11%2C11%2C31%2C11%2C31%2C21%2C11%2C21%2C11%2C11",
+        ],
+        [
+            {
+                "ccid": "C1991543728-POCLOUD",
+                "polygon": {
+                    "p1": [10, 10, 30, 10, 30, 20, 10, 20, 10, 10],
+                    "p2": [11, 11, 31, 11, 31, 21, 11, 21, 11, 11],
+                    "Union": True,
+                },
+            },
+            "concept_id=C1991543728-POCLOUD"
+            + "&polygon%5B%5D=10%2C10%2C30%2C10%2C30%2C20%2C10%2C20%2C10%2C10"
+            + "&polygon%5B%5D=11%2C11%2C31%2C11%2C31%2C21%2C11%2C21%2C11%2C11"
+            + "&options%5Bpolygon%5D%5Bor%5D=true",
+        ],
+        [
+            {
+                "ccid": ccid.split("=")[-1],
+                "line": [-0.37, -14.07, 4.75, 1.27, 25.13, -15.51],
+            },
+            ccid + "&line%5B%5D=-0.37%2C-14.07%2C4.75%2C1.27%2C25.13%2C-15.51",
+        ],
+        [
+            {
+                "ccid": ccid.split("=")[-1],
+                "line": {
+                    "line1": [-0.37, -14.07, 4.75, 1.27, 25.13, -15.51],
+                    "line2": [-1.37, -15.07, 5.75, 2.27, 26.13, -16.51],
+                },
+            },
+            ccid
+            + "&line%5B%5D=-0.37%2C-14.07%2C4.75%2C1.27%2C25.13%2C-15.51"
+            + "&line%5B%5D=-1.37%2C-15.07%2C5.75%2C2.27%2C26.13%2C-16.51",
+        ],
+        [
+            {
+                "ccid": ccid.split("=")[-1],
+                "line": {
+                    "line1": [-0.37, -14.07, 4.75, 1.27, 25.13, -15.51],
+                    "line2": [-1.37, -15.07, 5.75, 2.27, 26.13, -16.51],
+                    "Union": True,
+                },
+            },
+            ccid
+            + "&line%5B%5D=-0.37%2C-14.07%2C4.75%2C1.27%2C25.13%2C-15.51"
+            + "&line%5B%5D=-1.37%2C-15.07%2C5.75%2C2.27%2C26.13%2C-16.51"
+            + "&options%5Bline%5D%5Bor%5D=true",
+        ],
+        [
+            {
+                "ccid": ccid.split("=")[-1],
+                "circle": [-87.629717, 41.878112, 1000],
+            },
+            ccid + "&circle%5B%5D=-87.629717%2C41.878112%2C1000",
+        ],
+        [
+            {
+                "ccid": "C1991543728-POCLOUD",
+                "circle": {
+                    "c1": [-87.629717, 41.878112, 1000],
+                    "c2": [-75, 41.878112, 1000],
+                },
+            },
+            "concept_id=C1991543728-POCLOUD"
+            + "&circle%5B%5D=-87.629717%2C41.878112%2C1000"
+            + "&circle%5B%5D=-75%2C41.878112%2C1000",
+        ],
+        [
+            {
+                "ccid": "C1991543728-POCLOUD",
+                "circle": {
+                    "c1": [-87.629717, 41.878112, 1000],
+                    "c2": [-75, 41.878112, 1000],
+                    "Union": True,
+                },
+            },
+            "concept_id=C1991543728-POCLOUD"
+            + "&circle%5B%5D=-87.629717%2C41.878112%2C1000"
+            + "&circle%5B%5D=-75%2C41.878112%2C1000"
+            + "&options%5Bcircle%5D%5Bor%5D=true",
+        ],
+    ],
+)
+def test_get_cmr_urls(param, expected):
+    """Test that get_cmr_urls returns the correct urls."""
+    session = create_session(use_cache=True, cache_kwargs={"backend": "memory"})
+    cmr_urls = get_cmr_urls(**param, session=session)
+    assert isinstance(cmr_urls, list)
+    assert len(cmr_urls) > 0
+    cmr_url = "https://cmr.earthdata.nasa.gov/search/"
+    cached_urls = session.cache.urls()
+    if "doi" in param.keys():
+        # when searching by doi, first pydap searches for the concept_collection id
+        # and then searches for granules using that id.
+        # the first url is the collection search
+        url = cmr_url + "collections.json?doi="
+        doi_query = _quote(param["doi"]).replace("/", "%2F").replace("%2E", ".")
+        assert url + doi_query == cached_urls[0]
+        cached_urls = cached_urls[1:]  # remove the first url
+    cached_urls = cached_urls[0].split("?")[-1]  # get only the query part of the url
+    if "limit" not in param.keys():
+        # if page_size is not specified, it defaults to 50
+        expected += "&page_size=50"
+    assert set(expected.split("&")) == set(cached_urls.split("&"))