Repository: ecmwf/earthkit-data
Pull request #273 (2023-12-19 14:07:16)

Enable using url source as a stream
This PR enables using GRIB data from a URL or multiple URLs as a stream.

```python
import earthkit.data

ds = earthkit.data.from_source("url",
                       ["https://get.ecmwf.int/repository/test-data/earthkit-data/examples/test.grib",
                        "https://get.ecmwf.int/repository/test-data/earthkit-data/examples/test4.grib"], 
                        stream=True, batch_size=3)

for f in ds:
    # f is a fieldlist
    print(type(f))
    print(f"len={len(f)}")
    for g in f:
        print(f" {g}")
```
```
<class 'earthkit.data.readers.grib.memory.GribFieldListInMemory'>
len=3
 GribField(2t,None,20200513,1200,0,0)
 GribField(msl,None,20200513,1200,0,0)
 GribField(t,500,20070101,1200,0,0)
<class 'earthkit.data.readers.grib.memory.GribFieldListInMemory'>
len=3
 GribField(z,500,20070101,1200,0,0)
 GribField(t,850,20070101,1200,0,0)
 GribField(z,850,20070101,1200,0,0)
```

Example: https://earthkit-data--273.org.readthedocs.build/en/273/examples/grib_url_stream.html


Patch:
diff --git a/docs/examples.rst b/docs/examples.rst
index 1a9dd17..ad50dce 100644
--- a/docs/examples.rst
+++ b/docs/examples.rst
@@ -35,6 +35,7 @@ Here is a list of example notebooks to illustrate how to use earthkit-data.
     examples/grib_file_pattern.ipynb
     examples/grib_tar.ipynb
     examples/grib_url.ipynb
+    examples/grib_url_stream.ipynb
     examples/grib_to_netcdf.ipynb
     examples/numpy_fieldlist.ipynb
     examples/grib_nearest_gridpoint.ipynb
diff --git a/docs/examples/fdb.ipynb b/docs/examples/fdb.ipynb
index 2d8a0f8..acda9ae 100644
--- a/docs/examples/fdb.ipynb
+++ b/docs/examples/fdb.ipynb
@@ -161,7 +161,7 @@
    "id": "c2c8809e-9d53-4c69-af57-e3617e1df8b8",
    "metadata": {},
    "source": [
-    "When we use the **group_by** option `from_source()` gives us a stream iterator object. Each iteration step results in a Fieldlist object, which is built by consuming GRIB messages from the stream until the values of the metadata keys specified in *group_by* change. The generated Fieldlist keeps GRIB messages in memory then gets deleted when going out of scope."
+    "When we use the [group_by](../guide/sources.rst#fdb) option [from_source()](../guide/sources.rst#fdb) gives us a stream iterator object. Each iteration step results in a Fieldlist object, which is built by consuming GRIB messages from the stream until the values of the metadata keys specified in [group_by](../guide/sources.rst#fdb) change. The generated Fieldlist keeps GRIB messages in memory then gets deleted when going out of scope."
    ]
   },
   {
@@ -206,7 +206,7 @@
    "id": "standard-soviet",
    "metadata": {},
    "source": [
-    "We can read multiple fields into memory from the stream at a time by using **batch_size** in *from_source()*, Please note that *batch_size* cannot be used together with *group_by*."
+    "We can read multiple fields into memory from the stream at a time by using [batch_size](../guide/sources.rst#fdb) in [from_source()](../guide/sources.rst#fdb), Please note that *batch_size* cannot be used together with [group_by](../guide/sources.rst#fdb)."
    ]
   },
   {
@@ -252,7 +252,7 @@
    "id": "personal-sense",
    "metadata": {},
    "source": [
-    "When we use **batch_size=0** all the fields are loaded into memory and the resulting object iswill behave like a FieldList:"
+    "When we use [batch_size](../guide/sources.rst#fdb)=0 all the fields are loaded into memory and the resulting object iswill behave like a FieldList:"
    ]
   },
   {
@@ -1202,9 +1202,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "mpy38",
+   "display_name": "dev",
    "language": "python",
-   "name": "mpy38"
+   "name": "dev"
   },
   "language_info": {
    "codemirror_mode": {
@@ -1216,7 +1216,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.16"
+   "version": "3.10.13"
   }
  },
  "nbformat": 4,
diff --git a/docs/examples/grib_fdb_write.ipynb b/docs/examples/grib_fdb_write.ipynb
index 2eb9222..5a51b66 100644
--- a/docs/examples/grib_fdb_write.ipynb
+++ b/docs/examples/grib_fdb_write.ipynb
@@ -308,9 +308,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "pyfdb",
+   "display_name": "dev",
    "language": "python",
-   "name": "pyfdb"
+   "name": "dev"
   },
   "language_info": {
    "codemirror_mode": {
@@ -322,7 +322,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.12"
+   "version": "3.10.13"
   }
  },
  "nbformat": 4,
diff --git a/docs/examples/grib_from_stream.ipynb b/docs/examples/grib_from_stream.ipynb
index 341623e..8543fc3 100644
--- a/docs/examples/grib_from_stream.ipynb
+++ b/docs/examples/grib_from_stream.ipynb
@@ -26,7 +26,7 @@
    "source": [
     "earthkit-data can load GRIB data from a **stream**, which can be an FDB stream, a standard Python IO stream or any object implementing the necessary stream methods. \n",
     "\n",
-    "For simplicity, in this notebook we will use a **file stream** to demonstrate the usage of streams. First we ensure the example file is available."
+    "For simplicity, in this notebook we will use a **file stream** to demonstrate the usage of streams. First, we ensure the example file containing 6 messages is available."
    ]
   },
   {
@@ -70,7 +70,7 @@
    "id": "covered-greensboro",
    "metadata": {},
    "source": [
-    "We load it into earthkit-data by using the default settings (batch_size=1). With this when we iterate through *fs* it will consume one message from the stream at a time:"
+    "We load it into earthkit-data by using the default settings ([batch_size](../guide/sources.rst#stream)=1). With this when we iterate through *ds* it will consume one message from the stream at a time:"
    ]
   },
   {
@@ -80,7 +80,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "fs = earthkit.data.from_source(\"stream\", stream)"
+    "ds = earthkit.data.from_source(\"stream\", stream)"
    ]
   },
   {
@@ -111,7 +111,7 @@
     }
    ],
    "source": [
-    "for f in fs:\n",
+    "for f in ds:\n",
     "    # f is GribField object. It gets deleted when going out of scope\n",
     "    print(f)"
    ]
@@ -121,7 +121,7 @@
    "id": "brilliant-struggle",
    "metadata": {},
    "source": [
-    "Having finished the iteration there is no data available in *fs*.  We can close the stream:"
+    "Having finished the iteration there is no data available in *ds*.  We can close the stream:"
    ]
   },
   {
@@ -147,7 +147,7 @@
    "id": "3b7e9e3c-df32-44c6-aa4a-fcd2ff72b3b2",
    "metadata": {},
    "source": [
-    "When we use the **group_by** option `from_source()` gives us a stream iterator object. Each iteration step results in a Fieldlist object, which is built by consuming GRIB messages from the stream until the values of the metadata keys specified in *group_by* change. The generated Fieldlist keeps GRIB messages in memory then gets deleted when going out of scope."
+    "When we use the [group_by](../guide/sources.rst#stream) option [from_source()](../guide/sources.rst#stream) gives us a stream iterator object. Each iteration step results in a Fieldlist object, which is built by consuming GRIB messages from the stream until the values of the metadata keys specified in [group_by](../guide/sources.rst#stream) change. The generated Fieldlist keeps GRIB messages in memory then gets deleted when going out of scope."
    ]
   },
   {
@@ -158,7 +158,7 @@
    "outputs": [],
    "source": [
     "stream = open(\"test6.grib\", \"rb\")\n",
-    "fs = earthkit.data.from_source(\"stream\", stream, group_by=\"level\")"
+    "ds = earthkit.data.from_source(\"stream\", stream, group_by=\"level\")"
    ]
   },
   {
@@ -171,11 +171,11 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "<class 'earthkit.data.readers.grib.memory.FieldListInMemory'>\n",
+      "len=3\n",
       " GribField(t,1000,20180801,1200,0,0)\n",
       " GribField(u,1000,20180801,1200,0,0)\n",
       " GribField(v,1000,20180801,1200,0,0)\n",
-      "<class 'earthkit.data.readers.grib.memory.FieldListInMemory'>\n",
+      "len=3\n",
       " GribField(t,850,20180801,1200,0,0)\n",
       " GribField(u,850,20180801,1200,0,0)\n",
       " GribField(v,850,20180801,1200,0,0)\n"
@@ -183,8 +183,9 @@
     }
    ],
    "source": [
-    "for f in fs:\n",
-    "    print(type(f))\n",
+    "for f in ds:\n",
+    "    # f is a fieldlist\n",
+    "    print(f\"len={len(f)}\")\n",
     "    for g in f:\n",
     "        print(f\" {g}\")"
    ]
@@ -194,7 +195,7 @@
    "id": "89d33d4c-00d2-4f0b-996e-aaf5a5c9e161",
    "metadata": {},
    "source": [
-    "Having finished the iteration there is no data available in *fs*.  We can close the stream:"
+    "Having finished the iteration there is no data available in *ds*.  We can close the stream:"
    ]
   },
   {
@@ -220,7 +221,7 @@
    "id": "planned-weekly",
    "metadata": {},
    "source": [
-    "The **batch_size** option controls how many fields we read from the stream in one go. Please note that *batch_size* cannot be used together with *group_by*. In this example we create a stream and read 2 fields from it at a time by using **batch_size=2** in *from_source()*:"
+    "The [batch_size](../guide/sources.rst#stream)  option controls how many fields we read from the stream in one go. Please note that [batch_size](../guide/sources.rst#stream) cannot be used together with *group_by*. In this example we create a stream and read 2 fields from it at a time by using [batch_size](../guide/sources.rst#stream)=2 in [from_source()](../guide/sources.rst#stream):"
    ]
   },
   {
@@ -231,7 +232,7 @@
    "outputs": [],
    "source": [
     "stream = open(\"test6.grib\", \"rb\")\n",
-    "fs = earthkit.data.from_source(\"stream\", stream, batch_size=2)"
+    "ds = earthkit.data.from_source(\"stream\", stream, batch_size=2)"
    ]
   },
   {
@@ -244,21 +245,22 @@
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "<class 'earthkit.data.readers.grib.memory.FieldListInMemory'>\n",
+      "len=2\n",
       " GribField(t,1000,20180801,1200,0,0)\n",
       " GribField(u,1000,20180801,1200,0,0)\n",
-      "<class 'earthkit.data.readers.grib.memory.FieldListInMemory'>\n",
+      "len=2\n",
       " GribField(v,1000,20180801,1200,0,0)\n",
       " GribField(t,850,20180801,1200,0,0)\n",
-      "<class 'earthkit.data.readers.grib.memory.FieldListInMemory'>\n",
+      "len=2\n",
       " GribField(u,850,20180801,1200,0,0)\n",
       " GribField(v,850,20180801,1200,0,0)\n"
      ]
     }
    ],
    "source": [
-    "for f in fs:\n",
-    "    print(type(f))\n",
+    "for f in ds:\n",
+    "    # f is a fieldlist\n",
+    "    print(f\"len={len(f)}\")\n",
     "    for g in f:\n",
     "        print(f\" {g}\")"
    ]
@@ -268,7 +270,7 @@
    "id": "unavailable-actress",
    "metadata": {},
    "source": [
-    "Having finished the iteration there is no data available in *fs*.  We can close the stream:"
+    "Having finished the iteration there is no data available in *ds*.  We can close the stream:"
    ]
   },
   {
@@ -281,6 +283,46 @@
     "stream.close()"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "33798584-845d-4db8-9692-9fb0d5b24eec",
+   "metadata": {},
+   "source": [
+    "It is possible to set a batch size that is not a factor of the total number fields in the stream. In this case the last batch will simply contain less fields than prescribed by [batch_size](../guide/sources.rst#stream)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "id": "bf94a190-ec0e-4172-8e75-6518e48f50a4",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "len=4\n",
+      " GribField(t,1000,20180801,1200,0,0)\n",
+      " GribField(u,1000,20180801,1200,0,0)\n",
+      " GribField(v,1000,20180801,1200,0,0)\n",
+      " GribField(t,850,20180801,1200,0,0)\n",
+      "len=2\n",
+      " GribField(u,850,20180801,1200,0,0)\n",
+      " GribField(v,850,20180801,1200,0,0)\n"
+     ]
+    }
+   ],
+   "source": [
+    "stream = open(\"test6.grib\", \"rb\")\n",
+    "ds = earthkit.data.from_source(\"stream\", stream, batch_size=4)\n",
+    "\n",
+    "for f in ds:\n",
+    "    # f is a fieldlist\n",
+    "    print(f\"len={len(f)}\")\n",
+    "    for g in f:\n",
+    "        print(f\" {g}\")"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "permanent-uncertainty",
@@ -294,18 +336,18 @@
    "id": "blessed-paintball",
    "metadata": {},
    "source": [
-    "We can also set **batch_size=0** in *from_source()*:"
+    "We can also set [batch_size](../guide/sources.rst#stream)=0 in [from_source()](../guide/sources.rst#stream)."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 14,
    "id": "simple-london",
    "metadata": {},
    "outputs": [],
    "source": [
     "stream = open(\"test6.grib\", \"rb\")\n",
-    "fs = earthkit.data.from_source(\"stream\", stream, batch_size=0)"
+    "ds = earthkit.data.from_source(\"stream\", stream, batch_size=0)"
    ]
   },
   {
@@ -313,14 +355,14 @@
    "id": "seeing-freight",
    "metadata": {},
    "source": [
-    "The resulting earthkit-data object is empty at this point. However, as soon as we call any method on it it will consume the whole stream and load all the GRIB messages into memory. They will be stored in memory as long as *fs* exists.\n",
+    "The resulting earthkit-data object is empty at this point. However, as soon as we call any method on it it will consume the whole stream and load all the GRIB messages into memory. They will be stored in memory as long as *ds* exists.\n",
     "\n",
-    "We can call all the standard earthkit-data methods on *fs*:"
+    "We can call all the standard earthkit-data methods on *ds*:"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 14,
+   "execution_count": 15,
    "id": "meaning-oxide",
    "metadata": {},
    "outputs": [
@@ -330,18 +372,18 @@
        "6"
       ]
      },
-     "execution_count": 14,
+     "execution_count": 15,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "len(fs)"
+    "len(ds)"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 15,
+   "execution_count": 16,
    "id": "copyrighted-walnut",
    "metadata": {},
    "outputs": [
@@ -479,18 +521,18 @@
        "5       an       0  regular_ll  "
       ]
      },
-     "execution_count": 15,
+     "execution_count": 16,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "fs.ls()"
+    "ds.ls()"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 17,
    "id": "static-reasoning",
    "metadata": {},
    "outputs": [
@@ -568,19 +610,19 @@
        "1       an       0  regular_ll  "
       ]
      },
-     "execution_count": 16,
+     "execution_count": 17,
      "metadata": {},
      "output_type": "execute_result"
     }
    ],
    "source": [
-    "a = fs.sel(param=\"t\")\n",
+    "a = ds.sel(param=\"t\")\n",
     "a.ls()"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 17,
+   "execution_count": 18,
    "id": "spanish-wagon",
    "metadata": {},
    "outputs": [
@@ -957,7 +999,7 @@
        "  * number         (number) int64 0\n",
        "  * time           (time) datetime64[ns] 2018-08-01T12:00:00\n",
        "  * step           (step) timedelta64[ns] 00:00:00\n",
-       "  * isobaricInhPa  (isobaricInhPa) int64 1000 850\n",
+       "  * isobaricInhPa  (isobaricInhPa) float64 1e+03 850.0\n",
        "  * latitude       (latitude) float64 90.0 60.0 30.0 0.0 -30.0 -60.0 -90.0\n",
        "  * longitude      (longitude) float64 0.0 30.0 60.0 90.0 ... 270.0 300.0 330.0\n",
        "    valid_time     (time, step) datetime64[ns] ...\n",
@@ -970,9 +1012,9 @@
        "    GRIB_subCentre:          0\n",
        "    Conventions:             CF-1.7\n",
        "    institution:             European Centre for Medium-Range Weather Forecasts\n",
-       "    history:                 2023-06-08T12:46 GRIB to CDM+CF via cfgrib-0.9.1...</pre><div class='xr-wrap' style='display:none'><div class='xr-header'><div class='xr-obj-type'>xarray.Dataset</div></div><ul class='xr-sections'><li class='xr-section-item'><input id='section-731477d2-fe41-4adb-baed-a595dcfecd3d' class='xr-section-summary-in' type='checkbox' disabled ><label for='section-731477d2-fe41-4adb-baed-a595dcfecd3d' class='xr-section-summary'  title='Expand/collapse section'>Dimensions:</label><div class='xr-section-inline-details'><ul class='xr-dim-list'><li><span class='xr-has-index'>number</span>: 1</li><li><span class='xr-has-index'>time</span>: 1</li><li><span class='xr-has-index'>step</span>: 1</li><li><span class='xr-has-index'>isobaricInhPa</span>: 2</li><li><span class='xr-has-index'>latitude</span>: 7</li><li><span class='xr-has-index'>longitude</span>: 12</li></ul></div><div class='xr-section-details'></div></li><li class='xr-section-item'><input id='section-ce4e4447-10d3-4273-b2c5-5f00e4c31055' class='xr-section-summary-in' type='checkbox'  checked><label for='section-ce4e4447-10d3-4273-b2c5-5f00e4c31055' class='xr-section-summary' >Coordinates: <span>(7)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>number</span></div><div class='xr-var-dims'>(number)</div><div class='xr-var-dtype'>int64</div><div class='xr-var-preview xr-preview'>0</div><input id='attrs-08cf44d9-6432-462b-a5a0-c119d7867516' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-08cf44d9-6432-462b-a5a0-c119d7867516' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-504226e8-d5ea-46c0-85c6-48ca7cb8d6ca' class='xr-var-data-in' type='checkbox'><label for='data-504226e8-d5ea-46c0-85c6-48ca7cb8d6ca' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>ensemble member numerical id</dd><dt><span>units :</span></dt><dd>1</dd><dt><span>standard_name :</span></dt><dd>realization</dd></dl></div><div class='xr-var-data'><pre>array([0])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>time</span></div><div class='xr-var-dims'>(time)</div><div class='xr-var-dtype'>datetime64[ns]</div><div class='xr-var-preview xr-preview'>2018-08-01T12:00:00</div><input id='attrs-6afd3ea4-8459-4d4e-b3d8-8db82e558094' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-6afd3ea4-8459-4d4e-b3d8-8db82e558094' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-745a938d-73d2-4686-8f44-defe2fdcd06a' class='xr-var-data-in' type='checkbox'><label for='data-745a938d-73d2-4686-8f44-defe2fdcd06a' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>initial time of forecast</dd><dt><span>standard_name :</span></dt><dd>forecast_reference_time</dd></dl></div><div class='xr-var-data'><pre>array([&#x27;2018-08-01T12:00:00.000000000&#x27;], dtype=&#x27;datetime64[ns]&#x27;)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>step</span></div><div class='xr-var-dims'>(step)</div><div class='xr-var-dtype'>timedelta64[ns]</div><div class='xr-var-preview xr-preview'>00:00:00</div><input id='attrs-3726eb19-6c86-4f64-96a6-79732938e162' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-3726eb19-6c86-4f64-96a6-79732938e162' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-f568088e-ee0d-4337-a8a0-1a7ce591cb07' class='xr-var-data-in' type='checkbox'><label for='data-f568088e-ee0d-4337-a8a0-1a7ce591cb07' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>time since forecast_reference_time</dd><dt><span>standard_name :</span></dt><dd>forecast_period</dd></dl></div><div class='xr-var-data'><pre>array([0], dtype=&#x27;timedelta64[ns]&#x27;)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>isobaricInhPa</span></div><div class='xr-var-dims'>(isobaricInhPa)</div><div class='xr-var-dtype'>int64</div><div class='xr-var-preview xr-preview'>1000 850</div><input id='attrs-09e30367-a39b-46ef-a5c5-dec5576483af' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-09e30367-a39b-46ef-a5c5-dec5576483af' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-b56b4332-105c-4449-bd4c-34ac16ef187a' class='xr-var-data-in' type='checkbox'><label for='data-b56b4332-105c-4449-bd4c-34ac16ef187a' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>pressure</dd><dt><span>units :</span></dt><dd>hPa</dd><dt><span>positive :</span></dt><dd>down</dd><dt><span>stored_direction :</span></dt><dd>decreasing</dd><dt><span>standard_name :</span></dt><dd>air_pressure</dd></dl></div><div class='xr-var-data'><pre>array([1000,  850])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>latitude</span></div><div class='xr-var-dims'>(latitude)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>90.0 60.0 30.0 ... -60.0 -90.0</div><input id='attrs-54fe7756-c5ab-4527-b89c-8ffbde0b6276' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-54fe7756-c5ab-4527-b89c-8ffbde0b6276' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-a3192ca7-df31-49e6-ae7c-bc6dca7af65e' class='xr-var-data-in' type='checkbox'><label for='data-a3192ca7-df31-49e6-ae7c-bc6dca7af65e' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>units :</span></dt><dd>degrees_north</dd><dt><span>standard_name :</span></dt><dd>latitude</dd><dt><span>long_name :</span></dt><dd>latitude</dd><dt><span>stored_direction :</span></dt><dd>decreasing</dd></dl></div><div class='xr-var-data'><pre>array([ 90.,  60.,  30.,   0., -30., -60., -90.])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>longitude</span></div><div class='xr-var-dims'>(longitude)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>0.0 30.0 60.0 ... 270.0 300.0 330.0</div><input id='attrs-50142c43-0158-4973-81cd-0beb8201d4a8' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-50142c43-0158-4973-81cd-0beb8201d4a8' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-079d774c-142e-42f5-9cc0-d07f13d3c020' class='xr-var-data-in' type='checkbox'><label for='data-079d774c-142e-42f5-9cc0-d07f13d3c020' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>units :</span></dt><dd>degrees_east</dd><dt><span>standard_name :</span></dt><dd>longitude</dd><dt><span>long_name :</span></dt><dd>longitude</dd></dl></div><div class='xr-var-data'><pre>array([  0.,  30.,  60.,  90., 120., 150., 180., 210., 240., 270., 300., 330.])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>valid_time</span></div><div class='xr-var-dims'>(time, step)</div><div class='xr-var-dtype'>datetime64[ns]</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-8a14fb9b-6b9c-4864-a329-a8ed9f8e7b18' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-8a14fb9b-6b9c-4864-a329-a8ed9f8e7b18' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-d135f31a-0d81-4bb2-baf4-f7719500273a' class='xr-var-data-in' type='checkbox'><label for='data-d135f31a-0d81-4bb2-baf4-f7719500273a' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>standard_name :</span></dt><dd>time</dd><dt><span>long_name :</span></dt><dd>time</dd></dl></div><div class='xr-var-data'><pre>[1 values with dtype=datetime64[ns]]</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-1747c5fa-30d4-4f59-8764-8090930efe0e' class='xr-section-summary-in' type='checkbox'  checked><label for='section-1747c5fa-30d4-4f59-8764-8090930efe0e' class='xr-section-summary' >Data variables: <span>(1)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-var-name'><span>t</span></div><div class='xr-var-dims'>(number, time, step, isobaricInhPa, latitude, longitude)</div><div class='xr-var-dtype'>float32</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-28bce572-0820-4e27-8cb5-f1665add0f40' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-28bce572-0820-4e27-8cb5-f1665add0f40' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-0efb232b-92ed-4780-858f-7d87794678e9' class='xr-var-data-in' type='checkbox'><label for='data-0efb232b-92ed-4780-858f-7d87794678e9' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>GRIB_paramId :</span></dt><dd>130</dd><dt><span>GRIB_dataType :</span></dt><dd>an</dd><dt><span>GRIB_numberOfPoints :</span></dt><dd>84</dd><dt><span>GRIB_typeOfLevel :</span></dt><dd>isobaricInhPa</dd><dt><span>GRIB_stepUnits :</span></dt><dd>1</dd><dt><span>GRIB_stepType :</span></dt><dd>instant</dd><dt><span>GRIB_gridType :</span></dt><dd>regular_ll</dd><dt><span>GRIB_NV :</span></dt><dd>0</dd><dt><span>GRIB_Nx :</span></dt><dd>12</dd><dt><span>GRIB_Ny :</span></dt><dd>7</dd><dt><span>GRIB_cfName :</span></dt><dd>air_temperature</dd><dt><span>GRIB_cfVarName :</span></dt><dd>t</dd><dt><span>GRIB_gridDefinitionDescription :</span></dt><dd>Latitude/Longitude Grid</dd><dt><span>GRIB_iDirectionIncrementInDegrees :</span></dt><dd>30.0</dd><dt><span>GRIB_iScansNegatively :</span></dt><dd>0</dd><dt><span>GRIB_jDirectionIncrementInDegrees :</span></dt><dd>30.0</dd><dt><span>GRIB_jPointsAreConsecutive :</span></dt><dd>0</dd><dt><span>GRIB_jScansPositively :</span></dt><dd>0</dd><dt><span>GRIB_latitudeOfFirstGridPointInDegrees :</span></dt><dd>90.0</dd><dt><span>GRIB_latitudeOfLastGridPointInDegrees :</span></dt><dd>-90.0</dd><dt><span>GRIB_longitudeOfFirstGridPointInDegrees :</span></dt><dd>0.0</dd><dt><span>GRIB_longitudeOfLastGridPointInDegrees :</span></dt><dd>330.0</dd><dt><span>GRIB_missingValue :</span></dt><dd>9999</dd><dt><span>GRIB_name :</span></dt><dd>Temperature</dd><dt><span>GRIB_shortName :</span></dt><dd>t</dd><dt><span>GRIB_totalNumber :</span></dt><dd>0</dd><dt><span>GRIB_units :</span></dt><dd>K</dd><dt><span>long_name :</span></dt><dd>Temperature</dd><dt><span>units :</span></dt><dd>K</dd><dt><span>standard_name :</span></dt><dd>air_temperature</dd></dl></div><div class='xr-var-data'><pre>[168 values with dtype=float32]</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-054d3d08-8d5c-43a1-9f6f-09d2cc8737cb' class='xr-section-summary-in' type='checkbox'  ><label for='section-054d3d08-8d5c-43a1-9f6f-09d2cc8737cb' class='xr-section-summary' >Indexes: <span>(6)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-index-name'><div>number</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-847e3096-77fc-4edc-9a32-eea394fd3b30' class='xr-index-data-in' type='checkbox'/><label for='index-847e3096-77fc-4edc-9a32-eea394fd3b30' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(Int64Index([0], dtype=&#x27;int64&#x27;, name=&#x27;number&#x27;))</pre></div></li><li class='xr-var-item'><div class='xr-index-name'><div>time</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-1d591b5e-3d75-4486-aff1-c450a6fc075a' class='xr-index-data-in' type='checkbox'/><label for='index-1d591b5e-3d75-4486-aff1-c450a6fc075a' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(DatetimeIndex([&#x27;2018-08-01 12:00:00&#x27;], dtype=&#x27;datetime64[ns]&#x27;, name=&#x27;time&#x27;, freq=None))</pre></div></li><li class='xr-var-item'><div class='xr-index-name'><div>step</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-b3cb05a8-fb5a-4b45-852f-20183f2e48c4' class='xr-index-data-in' type='checkbox'/><label for='index-b3cb05a8-fb5a-4b45-852f-20183f2e48c4' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(TimedeltaIndex([&#x27;0 days&#x27;], dtype=&#x27;timedelta64[ns]&#x27;, name=&#x27;step&#x27;, freq=None))</pre></div></li><li class='xr-var-item'><div class='xr-index-name'><div>isobaricInhPa</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-7bec313e-f2b5-40d7-a4ec-ede7848f270b' class='xr-index-data-in' type='checkbox'/><label for='index-7bec313e-f2b5-40d7-a4ec-ede7848f270b' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(Int64Index([1000, 850], dtype=&#x27;int64&#x27;, name=&#x27;isobaricInhPa&#x27;))</pre></div></li><li class='xr-var-item'><div class='xr-index-name'><div>latitude</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-1380b445-9a3d-4543-852c-22c0ff619e76' class='xr-index-data-in' type='checkbox'/><label for='index-1380b445-9a3d-4543-852c-22c0ff619e76' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(Float64Index([90.0, 60.0, 30.0, 0.0, -30.0, -60.0, -90.0], dtype=&#x27;float64&#x27;, name=&#x27;latitude&#x27;))</pre></div></li><li class='xr-var-item'><div class='xr-index-name'><div>longitude</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-51ec8e35-6359-40fc-8b31-a9aeb950e96e' class='xr-index-data-in' type='checkbox'/><label for='index-51ec8e35-6359-40fc-8b31-a9aeb950e96e' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(Float64Index([0.0, 30.0, 60.0, 90.0, 120.0, 150.0, 180.0, 210.0, 240.0, 270.0,\n",
-       "              300.0, 330.0],\n",
-       "             dtype=&#x27;float64&#x27;, name=&#x27;longitude&#x27;))</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-f6e9d6eb-7d40-420c-9545-3849b69fa27a' class='xr-section-summary-in' type='checkbox'  checked><label for='section-f6e9d6eb-7d40-420c-9545-3849b69fa27a' class='xr-section-summary' >Attributes: <span>(7)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><dl class='xr-attrs'><dt><span>GRIB_edition :</span></dt><dd>1</dd><dt><span>GRIB_centre :</span></dt><dd>ecmf</dd><dt><span>GRIB_centreDescription :</span></dt><dd>European Centre for Medium-Range Weather Forecasts</dd><dt><span>GRIB_subCentre :</span></dt><dd>0</dd><dt><span>Conventions :</span></dt><dd>CF-1.7</dd><dt><span>institution :</span></dt><dd>European Centre for Medium-Range Weather Forecasts</dd><dt><span>history :</span></dt><dd>2023-06-08T12:46 GRIB to CDM+CF via cfgrib-0.9.10.3/ecCodes-2.27.1 with {&quot;source&quot;: &quot;N/A&quot;, &quot;filter_by_keys&quot;: {}, &quot;encode_cf&quot;: [&quot;parameter&quot;, &quot;time&quot;, &quot;geography&quot;, &quot;vertical&quot;]}</dd></dl></div></li></ul></div></div>"
+       "    history:                 2024-01-02T11:25 GRIB to CDM+CF via cfgrib-0.9.1...</pre><div class='xr-wrap' style='display:none'><div class='xr-header'><div class='xr-obj-type'>xarray.Dataset</div></div><ul class='xr-sections'><li class='xr-section-item'><input id='section-2b6366e3-a8a7-4329-afeb-8d7c60cd86d5' class='xr-section-summary-in' type='checkbox' disabled ><label for='section-2b6366e3-a8a7-4329-afeb-8d7c60cd86d5' class='xr-section-summary'  title='Expand/collapse section'>Dimensions:</label><div class='xr-section-inline-details'><ul class='xr-dim-list'><li><span class='xr-has-index'>number</span>: 1</li><li><span class='xr-has-index'>time</span>: 1</li><li><span class='xr-has-index'>step</span>: 1</li><li><span class='xr-has-index'>isobaricInhPa</span>: 2</li><li><span class='xr-has-index'>latitude</span>: 7</li><li><span class='xr-has-index'>longitude</span>: 12</li></ul></div><div class='xr-section-details'></div></li><li class='xr-section-item'><input id='section-a7eac6c6-aa02-4171-886e-e66d33d35f24' class='xr-section-summary-in' type='checkbox'  checked><label for='section-a7eac6c6-aa02-4171-886e-e66d33d35f24' class='xr-section-summary' >Coordinates: <span>(7)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>number</span></div><div class='xr-var-dims'>(number)</div><div class='xr-var-dtype'>int64</div><div class='xr-var-preview xr-preview'>0</div><input id='attrs-e33ea9dd-6e7b-42ae-8e7a-e54b2053841d' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-e33ea9dd-6e7b-42ae-8e7a-e54b2053841d' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-5059bdce-f75e-4b9c-8d41-789ab1dfaa5b' class='xr-var-data-in' type='checkbox'><label for='data-5059bdce-f75e-4b9c-8d41-789ab1dfaa5b' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>ensemble member numerical id</dd><dt><span>units :</span></dt><dd>1</dd><dt><span>standard_name :</span></dt><dd>realization</dd></dl></div><div class='xr-var-data'><pre>array([0])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>time</span></div><div class='xr-var-dims'>(time)</div><div class='xr-var-dtype'>datetime64[ns]</div><div class='xr-var-preview xr-preview'>2018-08-01T12:00:00</div><input id='attrs-50a7a563-83df-4a9c-a7f1-9c6a1dbbb921' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-50a7a563-83df-4a9c-a7f1-9c6a1dbbb921' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-63f86500-922e-49d2-bfbe-4f42cfe8f798' class='xr-var-data-in' type='checkbox'><label for='data-63f86500-922e-49d2-bfbe-4f42cfe8f798' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>initial time of forecast</dd><dt><span>standard_name :</span></dt><dd>forecast_reference_time</dd></dl></div><div class='xr-var-data'><pre>array([&#x27;2018-08-01T12:00:00.000000000&#x27;], dtype=&#x27;datetime64[ns]&#x27;)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>step</span></div><div class='xr-var-dims'>(step)</div><div class='xr-var-dtype'>timedelta64[ns]</div><div class='xr-var-preview xr-preview'>00:00:00</div><input id='attrs-14b996b3-cd4d-45a0-b8ac-ae26b0e52d40' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-14b996b3-cd4d-45a0-b8ac-ae26b0e52d40' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-64fd0845-29a1-409f-8088-47adfdb5ac56' class='xr-var-data-in' type='checkbox'><label for='data-64fd0845-29a1-409f-8088-47adfdb5ac56' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>time since forecast_reference_time</dd><dt><span>standard_name :</span></dt><dd>forecast_period</dd></dl></div><div class='xr-var-data'><pre>array([0], dtype=&#x27;timedelta64[ns]&#x27;)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>isobaricInhPa</span></div><div class='xr-var-dims'>(isobaricInhPa)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>1e+03 850.0</div><input id='attrs-c4af0352-49c7-4a64-be7e-b682e197c219' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-c4af0352-49c7-4a64-be7e-b682e197c219' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-7fc64dc7-6872-4c13-8655-bdeb8a7a3f7e' class='xr-var-data-in' type='checkbox'><label for='data-7fc64dc7-6872-4c13-8655-bdeb8a7a3f7e' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>pressure</dd><dt><span>units :</span></dt><dd>hPa</dd><dt><span>positive :</span></dt><dd>down</dd><dt><span>stored_direction :</span></dt><dd>decreasing</dd><dt><span>standard_name :</span></dt><dd>air_pressure</dd></dl></div><div class='xr-var-data'><pre>array([1000.,  850.])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>latitude</span></div><div class='xr-var-dims'>(latitude)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>90.0 60.0 30.0 ... -60.0 -90.0</div><input id='attrs-8ee8154b-5dc2-402f-83ab-8b3fbd5267db' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-8ee8154b-5dc2-402f-83ab-8b3fbd5267db' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-e848f2b9-dac3-438c-afd8-7e40a6c641b8' class='xr-var-data-in' type='checkbox'><label for='data-e848f2b9-dac3-438c-afd8-7e40a6c641b8' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>units :</span></dt><dd>degrees_north</dd><dt><span>standard_name :</span></dt><dd>latitude</dd><dt><span>long_name :</span></dt><dd>latitude</dd><dt><span>stored_direction :</span></dt><dd>decreasing</dd></dl></div><div class='xr-var-data'><pre>array([ 90.,  60.,  30.,   0., -30., -60., -90.])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>longitude</span></div><div class='xr-var-dims'>(longitude)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>0.0 30.0 60.0 ... 270.0 300.0 330.0</div><input id='attrs-ef413f70-07b8-42c1-892e-f1a3f57ebe26' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-ef413f70-07b8-42c1-892e-f1a3f57ebe26' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-257f8190-b5e9-49b3-8a81-36406f1d72be' class='xr-var-data-in' type='checkbox'><label for='data-257f8190-b5e9-49b3-8a81-36406f1d72be' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>units :</span></dt><dd>degrees_east</dd><dt><span>standard_name :</span></dt><dd>longitude</dd><dt><span>long_name :</span></dt><dd>longitude</dd></dl></div><div class='xr-var-data'><pre>array([  0.,  30.,  60.,  90., 120., 150., 180., 210., 240., 270., 300., 330.])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span>valid_time</span></div><div class='xr-var-dims'>(time, step)</div><div class='xr-var-dtype'>datetime64[ns]</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-17f1768b-794c-40c1-9a57-b8481f6c46ff' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-17f1768b-794c-40c1-9a57-b8481f6c46ff' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-81984098-190c-4d74-9157-9d2ecf2112c2' class='xr-var-data-in' type='checkbox'><label for='data-81984098-190c-4d74-9157-9d2ecf2112c2' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>standard_name :</span></dt><dd>time</dd><dt><span>long_name :</span></dt><dd>time</dd></dl></div><div class='xr-var-data'><pre>[1 values with dtype=datetime64[ns]]</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-bea46411-fb89-4567-b825-edc4ed7c5d50' class='xr-section-summary-in' type='checkbox'  checked><label for='section-bea46411-fb89-4567-b825-edc4ed7c5d50' class='xr-section-summary' >Data variables: <span>(1)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-var-name'><span>t</span></div><div class='xr-var-dims'>(number, time, step, isobaricInhPa, latitude, longitude)</div><div class='xr-var-dtype'>float32</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-cbe2446d-f0d7-4600-bd9c-c89a99ac5433' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-cbe2446d-f0d7-4600-bd9c-c89a99ac5433' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-d7167053-4c06-4230-8bd1-3e7c6bcad833' class='xr-var-data-in' type='checkbox'><label for='data-d7167053-4c06-4230-8bd1-3e7c6bcad833' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>GRIB_paramId :</span></dt><dd>130</dd><dt><span>GRIB_dataType :</span></dt><dd>an</dd><dt><span>GRIB_numberOfPoints :</span></dt><dd>84</dd><dt><span>GRIB_typeOfLevel :</span></dt><dd>isobaricInhPa</dd><dt><span>GRIB_stepUnits :</span></dt><dd>1</dd><dt><span>GRIB_stepType :</span></dt><dd>instant</dd><dt><span>GRIB_gridType :</span></dt><dd>regular_ll</dd><dt><span>GRIB_NV :</span></dt><dd>0</dd><dt><span>GRIB_Nx :</span></dt><dd>12</dd><dt><span>GRIB_Ny :</span></dt><dd>7</dd><dt><span>GRIB_cfName :</span></dt><dd>air_temperature</dd><dt><span>GRIB_cfVarName :</span></dt><dd>t</dd><dt><span>GRIB_gridDefinitionDescription :</span></dt><dd>Latitude/Longitude Grid</dd><dt><span>GRIB_iDirectionIncrementInDegrees :</span></dt><dd>30.0</dd><dt><span>GRIB_iScansNegatively :</span></dt><dd>0</dd><dt><span>GRIB_jDirectionIncrementInDegrees :</span></dt><dd>30.0</dd><dt><span>GRIB_jPointsAreConsecutive :</span></dt><dd>0</dd><dt><span>GRIB_jScansPositively :</span></dt><dd>0</dd><dt><span>GRIB_latitudeOfFirstGridPointInDegrees :</span></dt><dd>90.0</dd><dt><span>GRIB_latitudeOfLastGridPointInDegrees :</span></dt><dd>-90.0</dd><dt><span>GRIB_longitudeOfFirstGridPointInDegrees :</span></dt><dd>0.0</dd><dt><span>GRIB_longitudeOfLastGridPointInDegrees :</span></dt><dd>330.0</dd><dt><span>GRIB_missingValue :</span></dt><dd>9999</dd><dt><span>GRIB_name :</span></dt><dd>Temperature</dd><dt><span>GRIB_shortName :</span></dt><dd>t</dd><dt><span>GRIB_totalNumber :</span></dt><dd>0</dd><dt><span>GRIB_units :</span></dt><dd>K</dd><dt><span>long_name :</span></dt><dd>Temperature</dd><dt><span>units :</span></dt><dd>K</dd><dt><span>standard_name :</span></dt><dd>air_temperature</dd></dl></div><div class='xr-var-data'><pre>[168 values with dtype=float32]</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-879b4eb2-98be-4a35-8940-d7401dcd153f' class='xr-section-summary-in' type='checkbox'  ><label for='section-879b4eb2-98be-4a35-8940-d7401dcd153f' class='xr-section-summary' >Indexes: <span>(6)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-index-name'><div>number</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-e96e7475-ded6-4e66-b16d-2ff1b0566f9d' class='xr-index-data-in' type='checkbox'/><label for='index-e96e7475-ded6-4e66-b16d-2ff1b0566f9d' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(Index([0], dtype=&#x27;int64&#x27;, name=&#x27;number&#x27;))</pre></div></li><li class='xr-var-item'><div class='xr-index-name'><div>time</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-1f4de1ec-d805-4b58-9ac8-07d0af42a271' class='xr-index-data-in' type='checkbox'/><label for='index-1f4de1ec-d805-4b58-9ac8-07d0af42a271' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(DatetimeIndex([&#x27;2018-08-01 12:00:00&#x27;], dtype=&#x27;datetime64[ns]&#x27;, name=&#x27;time&#x27;, freq=None))</pre></div></li><li class='xr-var-item'><div class='xr-index-name'><div>step</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-d98c7386-5553-4ea4-ac0b-682239f9678e' class='xr-index-data-in' type='checkbox'/><label for='index-d98c7386-5553-4ea4-ac0b-682239f9678e' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(TimedeltaIndex([&#x27;0 days&#x27;], dtype=&#x27;timedelta64[ns]&#x27;, name=&#x27;step&#x27;, freq=None))</pre></div></li><li class='xr-var-item'><div class='xr-index-name'><div>isobaricInhPa</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-1988c873-2505-4f89-8e28-51160ad38a1d' class='xr-index-data-in' type='checkbox'/><label for='index-1988c873-2505-4f89-8e28-51160ad38a1d' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(Index([1000.0, 850.0], dtype=&#x27;float64&#x27;, name=&#x27;isobaricInhPa&#x27;))</pre></div></li><li class='xr-var-item'><div class='xr-index-name'><div>latitude</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-11b310dc-6319-4b19-ac01-96b99624bcd7' class='xr-index-data-in' type='checkbox'/><label for='index-11b310dc-6319-4b19-ac01-96b99624bcd7' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(Index([90.0, 60.0, 30.0, 0.0, -30.0, -60.0, -90.0], dtype=&#x27;float64&#x27;, name=&#x27;latitude&#x27;))</pre></div></li><li class='xr-var-item'><div class='xr-index-name'><div>longitude</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-c9c694fe-0537-4ce8-9c4b-9a57887113b3' class='xr-index-data-in' type='checkbox'/><label for='index-c9c694fe-0537-4ce8-9c4b-9a57887113b3' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(Index([0.0, 30.0, 60.0, 90.0, 120.0, 150.0, 180.0, 210.0, 240.0, 270.0, 300.0,\n",
+       "       330.0],\n",
+       "      dtype=&#x27;float64&#x27;, name=&#x27;longitude&#x27;))</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-d1e2fb57-df40-40db-9b0e-457fac32c1d8' class='xr-section-summary-in' type='checkbox'  checked><label for='section-d1e2fb57-df40-40db-9b0e-457fac32c1d8' class='xr-section-summary' >Attributes: <span>(7)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><dl class='xr-attrs'><dt><span>GRIB_edition :</span></dt><dd>1</dd><dt><span>GRIB_centre :</span></dt><dd>ecmf</dd><dt><span>GRIB_centreDescription :</span></dt><dd>European Centre for Medium-Range Weather Forecasts</dd><dt><span>GRIB_subCentre :</span></dt><dd>0</dd><dt><span>Conventions :</span></dt><dd>CF-1.7</dd><dt><span>institution :</span></dt><dd>European Centre for Medium-Range Weather Forecasts</dd><dt><span>history :</span></dt><dd>2024-01-02T11:25 GRIB to CDM+CF via cfgrib-0.9.10.4/ecCodes-2.32.1 with {&quot;source&quot;: &quot;N/A&quot;, &quot;filter_by_keys&quot;: {}, &quot;encode_cf&quot;: [&quot;parameter&quot;, &quot;time&quot;, &quot;geography&quot;, &quot;vertical&quot;]}</dd></dl></div></li></ul></div></div>"
       ],
       "text/plain": [
        "<xarray.Dataset>\n",
@@ -982,7 +1024,7 @@
        "  * number         (number) int64 0\n",
        "  * time           (time) datetime64[ns] 2018-08-01T12:00:00\n",
        "  * step           (step) timedelta64[ns] 00:00:00\n",
-       "  * isobaricInhPa  (isobaricInhPa) int64 1000 850\n",
+       "  * isobaricInhPa  (isobaricInhPa) float64 1e+03 850.0\n",
        "  * latitude       (latitude) float64 90.0 60.0 30.0 0.0 -30.0 -60.0 -90.0\n",
        "  * longitude      (longitude) float64 0.0 30.0 60.0 90.0 ... 270.0 300.0 330.0\n",
        "    valid_time     (time, step) datetime64[ns] ...\n",
@@ -995,10 +1037,10 @@
        "    GRIB_subCentre:          0\n",
        "    Conventions:             CF-1.7\n",
        "    institution:             European Centre for Medium-Range Weather Forecasts\n",
-       "    history:                 2023-06-08T12:46 GRIB to CDM+CF via cfgrib-0.9.1..."
+       "    history:                 2024-01-02T11:25 GRIB to CDM+CF via cfgrib-0.9.1..."
       ]
      },
-     "execution_count": 17,
+     "execution_count": 18,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -1018,7 +1060,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
+   "execution_count": 19,
    "id": "through-mistress",
    "metadata": {},
    "outputs": [],
@@ -1037,9 +1079,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "mpy38",
+   "display_name": "dev",
    "language": "python",
-   "name": "mpy38"
+   "name": "dev"
   },
   "language_info": {
    "codemirror_mode": {
@@ -1051,7 +1093,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.16"
+   "version": "3.10.13"
   }
  },
  "nbformat": 4,
diff --git a/docs/examples/grib_url.ipynb b/docs/examples/grib_url.ipynb
index c6e9a5c..4da393f 100644
--- a/docs/examples/grib_url.ipynb
+++ b/docs/examples/grib_url.ipynb
@@ -894,9 +894,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "mpy38",
+   "display_name": "dev",
    "language": "python",
-   "name": "mpy38"
+   "name": "dev"
   },
   "language_info": {
    "codemirror_mode": {
@@ -908,7 +908,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.16"
+   "version": "3.10.13"
   }
  },
  "nbformat": 4,
diff --git a/docs/examples/grib_url_stream.ipynb b/docs/examples/grib_url_stream.ipynb
new file mode 100644
index 0000000..02b1bd7
--- /dev/null
+++ b/docs/examples/grib_url_stream.ipynb
@@ -0,0 +1,186 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "ca9aa5f2-705a-4d80-b400-6acf7ff86786",
+   "metadata": {},
+   "source": [
+    "## Reading GRIB from an URL as a stream"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "c23ffb07-747c-4535-b608-66b080d6d4e5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import earthkit.data"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0a58377e-7be5-4951-8a76-0d5c24444898",
+   "metadata": {},
+   "source": [
+    "earthkit-data can read GRIB data from a URL as a stream without writing anything to disk. This can be activated with the **stream=True** kwarg when calling [from_source()](../guide/sources.rst#url) with a \"url\" source.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "1d2b07e3-1820-45ec-88d5-c78765533735",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "ds = earthkit.data.from_source(\"url\", \n",
+    "                       \"https://get.ecmwf.int/repository/test-data/earthkit-data/examples/test4.grib\", \n",
+    "                        stream=True)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "51964579-bf3f-4948-918d-fcd35581950e",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "for f in ds:\n",
+    "    # f is GribField object. It gets deleted when going out of scope\n",
+    "    print(f)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7f595ec8-008f-4198-8a3a-f8346fbfe09c",
+   "metadata": {},
+   "source": [
+    "### Stream options"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "994802cc-2e74-4af4-8e20-031f5937df62",
+   "metadata": {},
+   "source": [
+    "To control how the stream is read use [batch_size](../guide/sources.rst#url) and [group_by](../guide/sources.rst#url). E.g. the following code reads the GRIB data in messages of 2."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "b7c9a685-6b38-4236-ae74-a14445e8bb95",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "len=2\n",
+      " GribField(t,500,20070101,1200,0,0)\n",
+      " GribField(z,500,20070101,1200,0,0)\n",
+      "len=2\n",
+      " GribField(t,850,20070101,1200,0,0)\n",
+      " GribField(z,850,20070101,1200,0,0)\n"
+     ]
+    }
+   ],
+   "source": [
+    "ds = earthkit.data.from_source(\"url\", \n",
+    "                       \"https://get.ecmwf.int/repository/test-data/earthkit-data/examples/test4.grib\", \n",
+    "                        stream=True, batch_size=2)\n",
+    "\n",
+    "for f in ds:\n",
+    "    # f is a fieldlist\n",
+    "    print(f\"len={len(f)}\")\n",
+    "    for g in f:\n",
+    "        print(f\" {g}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cb7d74b5-73e0-4e57-9d80-e45a9dc725d8",
+   "metadata": {},
+   "source": [
+    "### Multiple URLs"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6d5efce5-c9d0-4edf-a2cf-312ced9339b5",
+   "metadata": {},
+   "source": [
+    "The stream option works even when the input is a list of URLs."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "891af3e5-5aa5-451f-b7bb-a86eaf7c8ffe",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "len=3\n",
+      " GribField(t,500,20070101,1200,0,0)\n",
+      " GribField(z,500,20070101,1200,0,0)\n",
+      " GribField(t,850,20070101,1200,0,0)\n",
+      "len=3\n",
+      " GribField(z,850,20070101,1200,0,0)\n",
+      " GribField(t,1000,20180801,1200,0,0)\n",
+      " GribField(u,1000,20180801,1200,0,0)\n",
+      "len=3\n",
+      " GribField(v,1000,20180801,1200,0,0)\n",
+      " GribField(t,850,20180801,1200,0,0)\n",
+      " GribField(u,850,20180801,1200,0,0)\n",
+      "len=1\n",
+      " GribField(v,850,20180801,1200,0,0)\n"
+     ]
+    }
+   ],
+   "source": [
+    "ds = earthkit.data.from_source(\"url\", \n",
+    "                       [\"https://get.ecmwf.int/repository/test-data/earthkit-data/examples/test4.grib\", \n",
+    "                       \"https://get.ecmwf.int/repository/test-data/earthkit-data/examples/test6.grib\"], \n",
+    "                        stream=True, batch_size=3)\n",
+    "\n",
+    "for f in ds:\n",
+    "    # f is a fieldlist\n",
+    "    print(f\"len={len(f)}\")\n",
+    "    for g in f:\n",
+    "        print(f\" {g}\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "d287fb70-66fb-4b2f-8c07-a3cca7c3035d",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "dev",
+   "language": "python",
+   "name": "dev"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.13"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/docs/guide/sources.rst b/docs/guide/sources.rst
index da540ce..17c7b20 100644
--- a/docs/guide/sources.rst
+++ b/docs/guide/sources.rst
@@ -67,10 +67,10 @@ file
 
   :param path: input path(s)
   :type path: str, list
-  :param bool expand_user: replaces the leading ~ or ~user in ``path`` by that user's home directory. See ``os.path.expanduser``
-  :param bool expand_vars:  expands shell environment variables in ``path``. See ``os.path.expandpath``
-  :param bool unix_glob: allows UNIX globbing in ``path``
-  :param bool recursive_glob: allows recursive scanning of directories. Only used when ``uxix_glob`` is True
+  :param bool expand_user: replace the leading ~ or ~user in ``path`` by that user's home directory. See ``os.path.expanduser``
+  :param bool expand_vars:  expand shell environment variables in ``path``. See ``os.path.expandpath``
+  :param bool unix_glob: allow UNIX globbing in ``path``
+  :param bool recursive_glob: allow recursive scanning of directories. Only used when ``uxix_glob`` is True
 
   *earthkit-data* will inspect the content of the files to check for any of the
   supported :ref:`data formats <data-format>`.
@@ -144,7 +144,7 @@ file-pattern
 url
 ---
 
-.. py:function:: from_source("url", url, unpack=True)
+.. py:function:: from_source("url", url, unpack=True, stream=False, batch_size=1, group_by=None)
   :noindex:
 
   The ``url`` source will download the data from the address specified and store it in the :ref:`cache <caching>`. The supported data formats are the same as for the :ref:`file <data-sources-file>` data source above.
@@ -152,16 +152,31 @@ url
   :param url: the URL to download
   :type url: str
   :param bool unpack: for archive formats such as ``.zip``, ``.tar``, ``.tar.gz``, etc, *earthkit-data* will attempt to open it and extract any usable file. To keep the downloaded file as is use ``unpack=False``
+  :param bool stream: when it is ``True`` the data is read as a stream. Otherwise the data is retrieved into a file and stored in the :ref:`cache <caching>`. This option only works for GRIB data. No archive formats supported (``unpack`` is ignored).
+  :param int batch_size: used when ``stream=True`` and ``group_by`` is unset. It defines how many GRIB messages are consumed from the stream and kept in memory at a time. For details see :ref:`stream source <data-sources-stream>`.
+  :param group_by: used when ``stream=True`` and can specify one or more metadata keys to control how GRIB messages are read from the stream. For details see :ref:`stream source <data-sources-stream>`.
+  :type group_by: str, list of str
+  :param dict **kwargs: other keyword arguments specifying the request
 
   .. code-block:: python
 
-      import earthkit.data
+      >>> import earthkit.data
+      >>> ds = earthkit.data.from_source(
+      ...     "url",
+      ...     "https://get.ecmwf.int/repository/test-data/earthkit-data/examples/test4.grib",
+      ... )
+      >>> len(ds)
+      4
+
+  Further examples:
 
-      ds = earthkit.data.from_source("url", "https://www.example.com/data.csv")
+    - :ref:`/examples/grib_url.ipynb`
+    - :ref:`/examples/grib_url_stream.ipynb`
 
 
 .. _data-sources-url-pattern:
 
+
 url-pattern
 -----------
 
@@ -205,13 +220,17 @@ url-pattern
 stream
 --------------
 
-.. py:function:: from_source("stream", stream, batch_size=1)
+.. py:function:: from_source("stream", stream, batch_size=1, group_by=None)
   :noindex:
 
   The ``stream`` will read data from a stream, which can be an FDB stream, a standard Python IO stream or any object implementing the necessary stream methods. At the moment it only works for GRIB data.
 
   :param stream: the stream
-  :param bool batch_size: defines how many GRIB messages are consumed from the stream and kept in memory at a time. ``batch_size=0`` means all the messages will be loaded and stored in memory. When ``batch_size`` is not zero ``from_source`` gives us a stream iterator object. During the iteration temporary objects are created for each message then get deleted when going out of scope.
+  :param int batch_size: used when ``group_by`` is unset. It defines how many GRIB messages are consumed from the stream and kept in memory at a time. ``batch_size=0`` means all the messages will be loaded and stored in memory.  When ``batch_size`` is not zero ``from_source`` gives us a stream iterator object. During the iteration temporary objects are created for each message then get deleted when going out of scope. Used when ``group_by`` is unset.
+  :param group_by: specify one or more metadata keys to control how GRIB messages are read from the stream. When it is set ``from_source`` gives us a stream iterator object. Each iteration step results in a Fieldlist object, which is built by consuming GRIB messages from the stream until the values of the ``group_by`` metadata keys change. The generated Fieldlist keeps GRIB messages in memory then gets deleted when going out of scope. When ``group_by`` is set ``batch_size`` is ignored.
+  :type group_by: str, list of str
+  :param dict **kwargs: other keyword arguments specifying the request
+
 
   In the examples below, for simplicity, we create a file stream from a :ref:`grib` file and read it as a "stream". By default (``batch_size=1``) we will consume one message at a time:
 
@@ -223,14 +242,36 @@ stream
 
       # f is a GribField
       >>> for f in ds:
+      ...     print(f)
+      ...
+      GribField(t,500,20070101,1200,0,0)
+      GribField(z,500,20070101,1200,0,0)
+      GribField(t,850,20070101,1200,0,0)
+      GribField(z,850,20070101,1200,0,0)
+
+
+  We can use ``group_by`` to read fields with a matching level. ``ds`` is still just an iterator, but ``f`` is now a :obj:`FieldList <data.readers.grib.index.FieldList>`:
+
+    .. code-block:: python
+
+      >>> import earthkit.data
+      >>> stream = open("docs/examples/test4.grib", "rb")
+      >>> ds = earthkit.data.from_source("stream", stream, group_by="level")
+      >>> for f in ds:
       ...     print(len(f))
+      ...     for g in f:
+      ...         print(f" {g}")
       ...
-      1
-      1
+      2
+       GribField(t,500,20070101,1200,0,0)
+       GribField(z,500,20070101,1200,0,0)
+      2
+       GribField(t,850,20070101,1200,0,0)
+       GribField(z,850,20070101,1200,0,0)
 
   We can use ``batch_size=2`` to read 2 messages at a time:
 
-  .. code-block:: python
+    .. code-block:: python
 
       >>> import earthkit.data
       >>> stream = open("docs/examples/test4.grib", "rb")
@@ -239,13 +280,19 @@ stream
       # f is a FieldList containing 2 GribFields
       >>> for f in ds:
       ...     print(len(f))
+      ...     for g in f:
+      ...         print(f" {g}")
       ...
       2
+       GribField(t,500,20070101,1200,0,0)
+       GribField(z,500,20070101,1200,0,0)
       2
+       GribField(t,850,20070101,1200,0,0)
+       GribField(z,850,20070101,1200,0,0)
 
   With ``batch_size=0`` the whole stream will be consumed resulting in a FieldList object storing all the messages in memory. **Use this option carefully!**
 
-  .. code-block:: python
+    .. code-block:: python
 
       >>> import earthkit.data
       >>> stream = open("docs/examples/test4.grib", "rb")
@@ -262,6 +309,7 @@ stream
 
     - :ref:`/examples/grib_from_stream.ipynb`
     - :ref:`/examples/fdb.ipynb`
+    - :ref:`/examples/grib_url_stream.ipynb`
 
 
 .. _data-sources-memory:
@@ -299,10 +347,10 @@ ads
 .. py:function:: from_source("ads", dataset, *args, **kwargs)
   :noindex:
 
-  The ``ads`` source accesses the `Copernicus Atmosphere Data Store`_ (ADS), using the cdsapi_ package. In addition to data retrieval, ``request`` also has post-processing options such as ``grid`` and ``area`` for regridding and sub-area extraction respectively.
+  The ``ads`` source accesses the `Copernicus Atmosphere Data Store`_ (ADS), using the cdsapi_ package. In addition to data retrieval, ``request`` also has post-processing options such as ``grid`` and ``area`` for re-gridding and sub-area extraction respectively.
 
   :param str dataset: the name of the ADS dataset
-  :param tuple *args: specifies the request as a dict
+  :param tuple *args: specify the request as a dict
   :param dict **kwargs: other keyword arguments specifying the request
 
   The following example retrieves CAMS global reanalysis GRIB data for 2 parameters:
@@ -340,7 +388,7 @@ cds
   The ``cds`` source accesses the `Copernicus Climate Data Store`_ (CDS), using the cdsapi_ package. In addition to data retrieval, ``request`` also has post-processing options such as ``grid`` and ``area`` for regridding and sub-area extraction respectively.
 
   :param str dataset: the name of the CDS dataset
-  :param tuple *args: specifies the request as a dict
+  :param tuple *args: specify the request as a dict
   :param dict **kwargs: other keyword arguments specifying the request
 
   The following example retrieves ERA5 reanalysis GRIB data for a subarea for 2 surface parameters:
@@ -379,7 +427,7 @@ ecmwf-open-data
 
   The ``ecmwf-open-data`` source provides access to the `ECMWF open data`_, which is a subset of ECMWF real-time forecast data made available to the public free of charge.  It uses the `ecmwf-opendata <https://github.com/ecmwf/ecmwf-opendata>`_ package.
 
-  :param tuple *args: specifies the request as a dict
+  :param tuple *args: specify the request as a dict
   :param dict **kwargs: other keyword arguments specifying the request
 
   Details about the request format can be found `here <https://github.com/ecmwf/ecmwf-opendata>`__.
@@ -407,16 +455,16 @@ ecmwf-open-data
 fdb
 ---
 
-.. py:function:: from_source("fdb", *args, stream=True, group_by=None, batch_size=1, **kwargs)
+.. py:function:: from_source("fdb", *args, stream=True,  batch_size=1, group_by=None, **kwargs)
   :noindex:
 
   The ``fdb`` source accesses the `FDB (Fields DataBase) <https://fields-database.readthedocs.io/en/latest/>`_, which is a domain-specific object store developed at ECMWF for storing, indexing and retrieving GRIB data. earthkit-data uses the `pyfdb <https://pyfdb.readthedocs.io/en/latest>`_ package to retrieve data from FDB.
 
   :param tuple *args: positional arguments specifying the request as a dict
   :param bool stream: when it is ``True`` the data is read as a stream. Otherwise the data is retrieved into a file and stored in the :ref:`cache <caching>`.
-  :param group_by: used when ``stream=True`` and can specify one or more metadata keys to control how GRIB messages are read from the stream. When it is set ``from_source`` gives us a stream iterator object. Each iteration step results in a Fieldlist object, which is built by consuming GRIB messages from the stream until the values of the ``group_by`` metadata keys change. The generated Fieldlist keeps GRIB messages in memory then gets deleted when going out of scope. When ``group_by`` is set ``batch_size`` cannot be used.
+  :param int batch_size: used when ``stream=True`` and ``group_by`` is unset. It defines how many GRIB messages are consumed from the stream and kept in memory at a time. For details see :ref:`stream source <data-sources-stream>`.
+  :param group_by: used when ``stream=True`` and can specify one or more metadata keys to control how GRIB messages are read from the stream. For details see :ref:`stream source <data-sources-stream>`.
   :type group_by: str, list of str
-  :param bool batch_size: used when ``stream=True`` and ``group_by`` is unset. It defines how many GRIB messages are consumed from the stream and kept in memory at a time. ``batch_size=0`` means all the messages will be loaded and stored in memory.  When ``batch_size`` is not zero ``from_source`` gives us a stream iterator object. During the iteration temporary objects are created for each message then get deleted when going out of scope.
   :param dict **kwargs: other keyword arguments specifying the request
 
   The following example retrieves analysis :ref:`grib` data for 3 surface parameters as stream.
@@ -543,7 +591,7 @@ polytope
   The ``polytope`` source accesses the `Polytope web services <https://polytope-client.readthedocs.io/en/latest/>`_ , using the polytope-client_ package.
 
   :param str collection: the name of the polytope collection
-  :param tuple *args: specifies the request as a dict
+  :param tuple *args: specify the request as a dict
   :param dict **kwargs: other keyword arguments specifying the request
 
   The following example retrieves GRIB data from the "ecmwf-mars" polytope collection:
@@ -590,7 +638,7 @@ wekeo
   `WEkEO`_ is the Copernicus DIAS reference service for environmental data and virtual processing environments. The ``wekeo`` source provides access to `WEkEO`_ using the WEkEO grammar. The retrieval is based on the hda_ Python API.
 
   :param str dataset: the name of the WEkEO dataset
-  :param tuple *args: specifies the request as a dict
+  :param tuple *args: specify the request as a dict
   :param dict **kwargs: other keyword arguments specifying the request
 
   The following example retrieves Normalized Difference Vegetation Index data derived from EO satellite imagery in NetCDF format:
@@ -635,7 +683,7 @@ wekeocds
   `WEkEO`_ is the Copernicus DIAS reference service for environmental data and virtual processing environments. The ``wekeocds`` source provides access to `Copernicus Climate Data Store`_ (CDS) datasets served on `WEkEO`_ using the `cdsapi`_ grammar. The retrieval is based on the hda_ Python API.
 
   :param str dataset: the name of the WEkEO dataset
-  :param tuple *args: specifies the request as a dict
+  :param tuple *args: specify the request as a dict
   :param dict **kwargs: other keyword arguments specifying the request
 
   The following example retrieves ERA5 surface data for multiple days in GRIB format:
diff --git a/earthkit/data/readers/__init__.py b/earthkit/data/readers/__init__.py
index 30fa545..cae4567 100644
--- a/earthkit/data/readers/__init__.py
+++ b/earthkit/data/readers/__init__.py
@@ -113,7 +113,7 @@ def _readers(method_name):
     return {k[0]: v for k, v in _READERS.items() if k[1] == method_name}
 
 
-def _find_reader(method_name, source, path_or_bufr_or_stream, magic):
+def _find_reader(method_name, source, path_or_bufr_or_stream, magic, **kwargs):
     """Helper function to create a reader.
 
     Tries all the registered methods stored in _READERS.
@@ -122,7 +122,7 @@ def _find_reader(method_name, source, path_or_bufr_or_stream, magic):
         # We do two passes, the second one
         # allow the plugin to look deeper in the buffer
         for name, r in _readers(method_name).items():
-            reader = r(source, path_or_bufr_or_stream, magic, deeper_check)
+            reader = r(source, path_or_bufr_or_stream, magic, deeper_check, **kwargs)
             if reader is not None:
                 return reader.mutate()
 
@@ -172,7 +172,7 @@ def memory_reader(source, buf):
     return _find_reader("memory_reader", source, buf, magic)
 
 
-def stream_reader(source, stream):
+def stream_reader(source, stream, memory):
     """Create a reader for a stream"""
     magic = None
     if hasattr(stream, "peek") and callable(stream.peek):
@@ -184,4 +184,4 @@ def stream_reader(source, stream):
         except Exception:
             pass
 
-    return _find_reader("stream_reader", source, stream, magic)
+    return _find_reader("stream_reader", source, stream, magic, memory=memory)
diff --git a/earthkit/data/readers/grib/__init__.py b/earthkit/data/readers/grib/__init__.py
index 5d14250..cb6a252 100644
--- a/earthkit/data/readers/grib/__init__.py
+++ b/earthkit/data/readers/grib/__init__.py
@@ -35,12 +35,12 @@ def memory_reader(source, buf, magic=None, deeper_check=False):
         return GribFieldListInMemory(source, GribMessageMemoryReader(buf))
 
 
-def stream_reader(source, stream, magic=None, deeper_check=False):
+def stream_reader(source, stream, magic=None, deeper_check=False, memory=False):
     # by default we assume the stream is grib data
     if magic is None or _match_magic(magic, deeper_check):
         from .memory import GribFieldListInMemory, GribStreamReader
 
         r = GribStreamReader(stream)
-        if not source.group_by and source.batch_size == 0:
+        if memory:
             r = GribFieldListInMemory(source, r)
         return r
diff --git a/earthkit/data/readers/grib/memory.py b/earthkit/data/readers/grib/memory.py
index 65170fa..2e2e9d5 100644
--- a/earthkit/data/readers/grib/memory.py
+++ b/earthkit/data/readers/grib/memory.py
@@ -51,7 +51,15 @@ class GribMemoryReader(Reader):
         return self._peeked
 
     def read_batch(self, n):
-        fields = [self.__next__() for _ in range(n)]
+        fields = []
+        for _ in range(n):
+            try:
+                fields.append(self.__next__())
+            except StopIteration:
+                break
+        if not fields:
+            raise StopIteration
+
         return GribFieldListInMemory.from_fields(fields)
 
     def read_group(self, group):
@@ -115,10 +123,14 @@ class GribStreamReader(GribMemoryReader):
 
     def __init__(self, stream):
         super().__init__()
-        self._stream = eccodes.StreamReader(stream)
+        self._stream = stream
+        self._reader = eccodes.StreamReader(stream)
+
+    def __del__(self):
+        self._stream.close()
 
     def _next_handle(self):
-        return self._stream._next_handle()
+        return self._reader._next_handle()
 
     def mutate(self):
         return self
@@ -169,13 +181,26 @@ class GribFieldListInMemory(GribFieldList, Reader):
         self._load()
         return len(self._fields)
 
-    def __getitem__(self, n):
+    def _getitem(self, n):
         self._load()
-        return self._fields[n]
+        if isinstance(n, int):
+            n = n if n >= 0 else len(self) + n
+            return self._fields[n]
 
     def _load(self):
         if not self._loaded:
-            for f in self._reader:
-                self._fields.append(f)
+            self._fields = [f for f in self._reader]
             self._loaded = True
             self._reader = None
+
+    def mutate_source(self):
+        return self
+
+    @classmethod
+    def merge(cls, readers):
+        assert all(isinstance(s, GribFieldListInMemory) for s in readers), readers
+        assert len(readers) > 1
+
+        from itertools import chain
+
+        return GribFieldListInMemory.from_fields(list(chain(*[f for f in readers])))
diff --git a/earthkit/data/sources/stream.py b/earthkit/data/sources/stream.py
index e8530ec..4d460ed 100644
--- a/earthkit/data/sources/stream.py
+++ b/earthkit/data/sources/stream.py
@@ -8,6 +8,7 @@
 #
 
 import logging
+from collections import deque
 
 from earthkit.data.readers import stream_reader
 from earthkit.data.sources.memory import MemoryBaseSource
@@ -17,81 +18,263 @@ from . import Source
 LOG = logging.getLogger(__name__)
 
 
-class StreamMemorySource(MemoryBaseSource):
-    def __init__(self, reader, **kwargs):
-        super().__init__(**kwargs)
-        self._reader = reader
+def parse_stream_kwargs(**kwargs):
+    group_by = kwargs.pop("group_by", None)
+    batch_size = kwargs.pop("batch_size", 1)
+    batch_size, group_by = check_stream_kwargs(batch_size, group_by)
+    stream_kwargs = dict(batch_size=batch_size, group_by=group_by)
+    return (stream_kwargs, kwargs)
 
-    def __iter__(self):
-        return iter(self._reader)
 
+def check_stream_kwargs(batch_size, group_by):
+    if group_by is None:
+        group_by = []
 
-class StreamSource(Source):
-    def __init__(self, stream, group_by=None, **kwargs):
-        super().__init__()
-        self._stream = stream
-        self._reader_ = None
-        self._group_by = group_by if group_by is not None else []
+    if isinstance(group_by, str):
+        group_by = [group_by]
+    else:
+        try:
+            group_by = list(group_by)
+        except Exception:
+            raise TypeError(f"unsupported types in group_by={group_by}")
 
-        if isinstance(self._group_by, str):
-            self._group_by = [self._group_by]
-        else:
-            try:
-                self._group_by = list(self._group_by)
-            except Exception:
-                raise TypeError(f"unsupported types in group_by={self._group_by}")
+    if group_by and not all([isinstance(x, str) for x in group_by]):
+        raise TypeError(f"group_by={group_by} must contain str values")
 
-        if self._group_by and not all([isinstance(x, str) for x in self._group_by]):
-            raise TypeError(f"`group_by`={self._group_by} must contain str values")
+    if batch_size is None:
+        batch_size = 1
 
-        if self._group_by and "batch_size" in kwargs:
-            raise TypeError(
-                "got an invalid keyword argument. batch_size cannot be used when group_by is set"
-            )
+    if batch_size < 0:
+        raise ValueError(f"batch_size={batch_size} cannot be negative")
 
-        self._batch_size = kwargs.pop("batch_size", 1)
-        if self._batch_size is None:
-            self._batch_size = 1
+    return (batch_size, group_by)
 
-        if self._batch_size < 0:
-            raise ValueError(f"batch_size={self._batch_size} cannot be negative")
-        if kwargs:
-            raise TypeError(f"got invalid keyword argument(s): {list(kwargs.keys())}")
 
-    @property
-    def batch_size(self):
-        return self._batch_size
+class StreamMemorySource(MemoryBaseSource):
+    def __init__(self, stream, **kwargs):
+        super().__init__(**kwargs)
+        self._stream = stream
+        self._reader_ = None
 
     @property
-    def group_by(self):
-        return self._group_by
+    def _reader(self):
+        if self._reader_ is None:
+            self._reader_ = stream_reader(self, self._stream, True)
+            if self._reader_ is None:
+                raise TypeError(f"could not create reader for stream={self._stream}")
+        return self._reader_
 
     def mutate(self):
-        if self.batch_size == 0:
-            source = StreamMemorySource(self._reader)
+        source = self._reader.mutate_source()
+        if source not in (None, self):
+            source._parent = self
             return source
-        else:
-            return self
+        return self
+
+
+class StreamSourceBase(Source):
+    def __init__(self, stream, *, batch_size=1, group_by=None):
+        super().__init__()
+        self._reader_ = None
+        self._stream = stream
+        self.batch_size, self.group_by = check_stream_kwargs(batch_size, group_by)
 
     def __iter__(self):
         return self
 
-    def __next__(self):
-        assert self.batch_size > 0
-        if self.group_by:
-            return self._reader.read_group(self.group_by)
-        else:
-            if self.batch_size == 1:
-                return self._reader.__next__()
-            else:
-                return self._reader.read_batch(self.batch_size)
+    def mutate(self):
+        return self
 
     @property
     def _reader(self):
         if self._reader_ is None:
-            self._reader_ = stream_reader(self, self._stream)
-            self._stream = None
+            self._reader_ = stream_reader(self, self._stream, False)
+            if self._reader_ is None:
+                raise TypeError(f"could not create reader for stream={self._stream}")
         return self._reader_
 
 
+class StreamSingleSource(StreamSourceBase):
+    def __init__(self, stream, **kwargs):
+        super().__init__(stream, **kwargs)
+        assert self.batch_size == 1
+
+    def __next__(self):
+        return self._reader.__next__()
+
+
+class StreamBatchSource(StreamSourceBase):
+    def __init__(self, stream, **kwargs):
+        super().__init__(stream, **kwargs)
+        assert self.batch_size > 1
+
+    def __next__(self):
+        return self._get_batch(self.batch_size)
+
+    def _get_batch(self, n):
+        return self._reader.read_batch(n)
+
+
+class StreamGroupSource(StreamSourceBase):
+    def __init__(self, stream, **kwargs):
+        super().__init__(stream, **kwargs)
+        assert self.group_by
+
+    def __next__(self):
+        return self._reader.read_group(self.group_by)
+
+
+class MultiStreamSource(Source):
+    def __init__(self, sources, group_by=None, batch_size=1):
+        self.sources = sources
+        if not isinstance(self.sources, deque):
+            self.sources = deque(self.sources)
+        self.group_by = group_by
+        self.batch_size = batch_size
+        self.current = None
+
+    def mutate(self):
+        if not self.group_by:
+            if self.batch_size == 0:
+                from .multi import MultiSource
+
+                return MultiSource([s() for s in self.sources])
+            elif self.batch_size > 1:
+                return MultiStreamBatchSource(
+                    self.sources, batch_size=self.batch_size, group_by=self.group_by
+                )
+
+        return self
+
+    def __iter__(self):
+        return self
+
+    def _next_source(self):
+        try:
+            return self.sources.popleft()()
+        except IndexError:
+            self.sources.clear()
+            pass
+
+    def __next__(self):
+        if self.current is None:
+            self.current = self._next_source()
+            if self.current is None:
+                raise StopIteration
+
+        try:
+            return self.current.__next__()
+        except StopIteration:
+            self.current = self._next_source()
+            if self.current is None:
+                raise StopIteration
+            return self.current.__next__()
+
+
+class MultiStreamBatchSource(MultiStreamSource):
+    def mutate(self):
+        return self
+
+    def __next__(self):
+        if self.current is None:
+            if self.sources:
+                self.current = self._next_source()
+            else:
+                raise StopIteration
+
+        delta = self.batch_size
+        try:
+            r = self.current._get_batch(self.batch_size)
+            delta -= len(r)
+        except StopIteration:
+            r = None
+
+        while delta > 0:
+            self.current = self._next_source()
+            if self.current is None:
+                break
+            else:
+                try:
+                    r1 = self.current._get_batch(delta)
+                    assert r1 is not None
+                    assert len(r1) > 0
+                    r = r + r1 if r is not None else r1
+                    assert len(r) > 0
+                    delta = self.batch_size - len(r)
+                except StopIteration:
+                    break
+
+        if r is None or len(r) == 0:
+            raise StopIteration
+
+        return r
+
+
+class StreamSource(StreamSourceBase):
+    def __init__(self, stream, **kwargs):
+        super().__init__(stream, **kwargs)
+
+        # print(f"kwargs={kwargs} {id(kwargs)}")
+        # if kwargs:
+        #     raise TypeError(f"got invalid keyword argument(s): {list(kwargs.keys())}")
+
+    def mutate(self):
+        assert self._reader_ is None
+
+        return make_source(
+            self._stream, batch_size=self.batch_size, group_by=self.group_by
+        )
+
+
+class StreamSourceMaker:
+    def __init__(self, create, data, stream_kwargs):
+        self.create = create
+        self.data = data
+        self.stream_kwargs = dict(stream_kwargs)
+        self.source = None
+
+    def __call__(self):
+        if self.source is None:
+            stream = self.create(self.data)
+            self.source = make_source(stream, **self.stream_kwargs)
+
+            prev = None
+            src = self.source
+            while src is not prev:
+                prev = src
+                src = src.mutate()
+            self.source = src
+
+        return self.source
+
+
+def make_source(stream, group_by, batch_size):
+    _kwargs = dict(batch_size=batch_size, group_by=group_by)
+
+    if group_by:
+        return StreamGroupSource(stream, **_kwargs)
+    elif batch_size == 0:
+        return StreamMemorySource(stream)
+    elif batch_size > 1:
+        return StreamBatchSource(stream, **_kwargs)
+    elif batch_size == 1:
+        return StreamSingleSource(stream, **_kwargs)
+
+    raise ValueError(f"Unsupported stream parameters {batch_size=} {group_by=}")
+
+
+def make_stream_from_method(owner, create, data, **kwargs):
+    stream_kwargs, kwargs = parse_stream_kwargs(**kwargs)
+
+    if kwargs:
+        raise TypeError(f"got invalid keyword argument(s): {list(kwargs.keys())}")
+
+    if not isinstance(data, (list, tuple)):
+        maker = StreamSourceMaker(create, data, stream_kwargs)
+        return maker()
+    else:
+        sources = [StreamSourceMaker(create, d, stream_kwargs) for d in data]
+        return MultiStreamSource(sources, **stream_kwargs)
+
+
 source = StreamSource
diff --git a/earthkit/data/sources/url.py b/earthkit/data/sources/url.py
index 07109c9..6b60835 100644
--- a/earthkit/data/sources/url.py
+++ b/earthkit/data/sources/url.py
@@ -117,6 +117,8 @@ class Url(FileSource):
         http_headers=None,
         update_if_out_of_date=False,
         fake_headers=None,  # When HEAD is not allowed but you know the size
+        stream=False,
+        **kwargs,
     ):
         super().__init__(filter=filter, merger=merger)
 
@@ -125,49 +127,72 @@ class Url(FileSource):
 
         self.url = url
         self.parts = parts
+        self.stream = stream
         LOG.debug("URL %s", url)
 
-        self.update_if_out_of_date = update_if_out_of_date
-
-        self.downloader = Downloader(
-            url,
-            chunk_size=chunk_size,
-            timeout=SETTINGS.get("url-download-timeout"),
-            verify=verify,
-            parts=parts,
-            range_method=range_method,
-            http_headers=http_headers,
-            fake_headers=fake_headers,
-            statistics_gatherer=record_statistics,
-            progress_bar=progress_bar,
-            resume_transfers=True,
-            override_target_file=False,
-            download_file_extension=".download",
-        )
-
-        if extension and extension[0] != ".":
-            extension = "." + extension
-
-        if extension is None:
-            extension = self.downloader.extension()
-
-        self.path = self.downloader.local_path()
-        if self.path is not None:
-            return
-
-        if force is None:
-            force = self.out_of_date
-
-        def download(target, _):
-            self.downloader.download(target)
-            return self.downloader.cache_data()
-
-        self.path = self.cache_file(
-            download,
-            dict(url=url, parts=parts),
-            extension=extension,
-            force=force,
-        )
+        if not self.stream:
+            self.update_if_out_of_date = update_if_out_of_date
+
+            self.downloader = Downloader(
+                url,
+                chunk_size=chunk_size,
+                timeout=SETTINGS.get("url-download-timeout"),
+                verify=verify,
+                parts=parts,
+                range_method=range_method,
+                http_headers=http_headers,
+                fake_headers=fake_headers,
+                statistics_gatherer=record_statistics,
+                progress_bar=progress_bar,
+                resume_transfers=True,
+                override_target_file=False,
+                download_file_extension=".download",
+            )
+
+            if extension and extension[0] != ".":
+                extension = "." + extension
+
+            if extension is None:
+                extension = self.downloader.extension()
+
+            self.path = self.downloader.local_path()
+            if self.path is not None:
+                return
+
+            if force is None:
+                force = self.out_of_date
+
+            def download(target, _):
+                self.downloader.download(target)
+                return self.downloader.cache_data()
+
+            print(f"{url=}")
+
+            self.path = self.cache_file(
+                download,
+                dict(url=url, parts=parts),
+                extension=extension,
+                force=force,
+            )
+        else:
+            self._kwargs = kwargs
+
+    def mutate(self):
+        if self.stream:
+
+            def url_to_stream(url):
+                from urllib.request import urlopen
+
+                # TODO: ensure stream is closed when consumed
+                return urlopen(url)
+
+            from .stream import make_stream_from_method
+
+            return make_stream_from_method(
+                self, url_to_stream, self.url, **self._kwargs
+            )
+        else:
+            return super().mutate()
 
     def connect_to_mirror(self, mirror):
         return mirror.connection_for_url(self, self.url, self.parts)
diff --git a/earthkit/data/utils/__init__.py b/earthkit/data/utils/__init__.py
index a565158..ed60fc4 100644
--- a/earthkit/data/utils/__init__.py
+++ b/earthkit/data/utils/__init__.py
@@ -167,3 +167,11 @@ def progress_bar(*, total=None, iterable=None, initial=0, desc=None):
         desc=desc,
         # dynamic_ncols=True, # make this the default?
     )
+
+
+def ensure_iterable(obj):
+    import collections.abc
+
+    if isinstance(obj, str) or not isinstance(obj, collections.abc.Iterable):
+        return [obj]
+    return obj


Test patch:
diff --git a/tests/grib/test_grib_stream.py b/tests/grib/test_grib_stream.py
index ae49662..508daa0 100644
--- a/tests/grib/test_grib_stream.py
+++ b/tests/grib/test_grib_stream.py
@@ -21,32 +21,34 @@ def repeat_list_items(items, count):
     return sum([[x] * count for x in items], [])
 
 
-def test_grib_from_stream_invalid_args():
-    with open(earthkit_examples_file("test6.grib"), "rb") as stream:
-        with pytest.raises(TypeError):
-            from_source("stream", stream, order_by="level")
-
-    with open(earthkit_examples_file("test6.grib"), "rb") as stream:
-        with pytest.raises(TypeError):
-            from_source("stream", stream, group_by=1)
-
-    with open(earthkit_examples_file("test6.grib"), "rb") as stream:
-        with pytest.raises(TypeError):
-            from_source("stream", stream, group_by=["level", 1])
-
-    with open(earthkit_examples_file("test6.grib"), "rb") as stream:
-        with pytest.raises(TypeError):
-            from_source("stream", stream, group_by="level", batch_size=1)
-
+@pytest.mark.parametrize(
+    "_kwargs,error",
+    [
+        (dict(order_by="level"), TypeError),
+        (dict(group_by=1), TypeError),
+        (dict(group_by=["level", 1]), TypeError),
+        # (dict(group_by="level", batch_size=1), TypeError),
+        (dict(batch_size=-1), ValueError),
+    ],
+)
+def test_grib_from_stream_invalid_args(_kwargs, error):
     with open(earthkit_examples_file("test6.grib"), "rb") as stream:
-        with pytest.raises(ValueError):
-            from_source("stream", stream, batch_size=-1)
+        with pytest.raises(error):
+            from_source("stream", stream, **_kwargs)
 
 
-@pytest.mark.parametrize("group_by", ["level", ["level", "gridType"]])
-def test_grib_from_stream_group_by(group_by):
+@pytest.mark.parametrize(
+    "_kwargs",
+    [
+        {"group_by": "level"},
+        {"group_by": "level", "batch_size": 0},
+        {"group_by": "level", "batch_size": 1},
+        {"group_by": ["level", "gridType"]},
+    ],
+)
+def test_grib_from_stream_group_by(_kwargs):
     with open(earthkit_examples_file("test6.grib"), "rb") as stream:
-        fs = from_source("stream", stream, group_by=group_by)
+        fs = from_source("stream", stream, **_kwargs)
 
         # no methods are available
         with pytest.raises(TypeError):
@@ -104,9 +106,16 @@ def test_grib_from_stream_group_by_convert_to_numpy(convert_kwargs, expected_sha
         assert sum([1 for _ in ds]) == 0
 
 
-def test_grib_from_stream_single_batch():
+@pytest.mark.parametrize(
+    "_kwargs",
+    [
+        {},
+        {"batch_size": 1},
+    ],
+)
+def test_grib_from_stream_single_batch(_kwargs):
     with open(earthkit_examples_file("test6.grib"), "rb") as stream:
-        ds = from_source("stream", stream)
+        ds = from_source("stream", stream, **_kwargs)
 
         # no fieldlist methods are available
         with pytest.raises(TypeError):
@@ -128,21 +137,27 @@ def test_grib_from_stream_single_batch():
         assert sum([1 for _ in ds]) == 0
 
 
-def test_grib_from_stream_multi_batch():
+@pytest.mark.parametrize(
+    "_kwargs,expected_meta",
+    [
+        ({"batch_size": 3}, [["t", "u", "v"], ["t", "u", "v"]]),
+        ({"batch_size": 4}, [["t", "u", "v", "t"], ["u", "v"]]),
+    ],
+)
+def test_grib_from_stream_multi_batch(_kwargs, expected_meta):
     with open(earthkit_examples_file("test6.grib"), "rb") as stream:
-        fs = from_source("stream", stream, batch_size=2)
+        ds = from_source("stream", stream, **_kwargs)
 
         # no methods are available
         with pytest.raises(TypeError):
-            len(fs)
+            len(ds)
 
-        ref = [["t", "u"], ["v", "t"], ["u", "v"]]
-        for i, f in enumerate(fs):
-            assert len(f) == 2
-            f.metadata("param") == ref[i]
+        for i, f in enumerate(ds):
+            assert len(f) == len(expected_meta[i])
+            f.metadata("param") == expected_meta[i]
 
         # stream consumed, no data is available
-        assert sum([1 for _ in fs]) == 0
+        assert sum([1 for _ in ds]) == 0
 
 
 @pytest.mark.parametrize(
@@ -196,20 +211,27 @@ def test_grib_from_stream_in_memory():
         assert len(ds) == 6
 
         expected_shape = (6, 7, 12)
-        ref = ["t", "u", "v", "t", "u", "v"]
+        md_ref = [
+            ("t", 1000),
+            ("u", 1000),
+            ("v", 1000),
+            ("t", 850),
+            ("u", 850),
+            ("v", 850),
+        ]
         val = []
 
         # iteration
         for f in ds:
-            v = f.metadata("param")
+            v = f.metadata(("param", "level"))
             val.append(v)
 
-        assert val == ref, "iteration"
+        assert val == md_ref, "iteration"
 
         # metadata
         val = []
-        val = ds.metadata("param")
-        assert val == ref, "method"
+        val = ds.metadata(("param", "level"))
+        assert val == md_ref, "method"
 
         # data
         assert ds.to_numpy().shape == expected_shape
@@ -228,6 +250,25 @@ def test_grib_from_stream_in_memory():
         vals = ds.to_numpy()[:, 0, 0]
         assert np.allclose(vals, ref)
 
+        # slicing
+        r = ds[0:3]
+        assert len(r) == 3
+        val = r.metadata(("param", "level"))
+        assert val == md_ref[0:3]
+
+        r = ds[-2:]
+        assert len(r) == 2
+        val = r.metadata(("param", "level"))
+        assert val == md_ref[-2:]
+
+        r = ds.sel(param="t")
+        assert len(r) == 2
+        val = r.metadata(("param", "level"))
+        assert val == [
+            ("t", 1000),
+            ("t", 850),
+        ]
+
 
 @pytest.mark.parametrize(
     "convert_kwargs,expected_shape",
diff --git a/tests/grib/test_grib_url_stream.py b/tests/grib/test_grib_url_stream.py
new file mode 100644
index 0000000..abdd03a
--- /dev/null
+++ b/tests/grib/test_grib_url_stream.py
@@ -0,0 +1,348 @@
+#!/usr/bin/env python3
+
+# (C) Copyright 2020 ECMWF.
+#
+# This software is licensed under the terms of the Apache Licence Version 2.0
+# which can be obtained at http://www.apache.org/licenses/LICENSE-2.0.
+# In applying this licence, ECMWF does not waive the privileges and immunities
+# granted to it by virtue of its status as an intergovernmental organisation
+# nor does it submit to any jurisdiction.
+#
+
+import numpy as np
+import pytest
+
+from earthkit.data import from_source
+from earthkit.data.core.temporary import temp_file
+from earthkit.data.testing import earthkit_remote_test_data_file
+
+
+def repeat_list_items(items, count):
+    return sum([[x] * count for x in items], [])
+
+
+@pytest.mark.parametrize(
+    "_kwargs,error",
+    [
+        (dict(order_by="level"), TypeError),
+        (dict(group_by=1), TypeError),
+        (dict(group_by=["level", 1]), TypeError),
+        # (dict(group_by="level", batch_size=1), TypeError),
+        (dict(batch_size=-1), ValueError),
+    ],
+)
+def test_grib_url_stream_invalid_args(_kwargs, error):
+    with pytest.raises(error):
+        from_source(
+            "url",
+            earthkit_remote_test_data_file("examples/test6.grib"),
+            stream=True,
+            **_kwargs,
+        )
+
+
+@pytest.mark.parametrize(
+    "_kwargs",
+    [
+        {"group_by": "level"},
+        {"group_by": "level", "batch_size": 0},
+        {"group_by": "level", "batch_size": 1},
+        {"group_by": ["level", "gridType"]},
+    ],
+)
+def test_grib_url_stream_group_by(_kwargs):
+    fs = from_source(
+        "url",
+        earthkit_remote_test_data_file("examples/test6.grib"),
+        stream=True,
+        **_kwargs,
+    )
+
+    # no methods are available
+    with pytest.raises(TypeError):
+        len(fs)
+
+    ref = [
+        [("t", 1000), ("u", 1000), ("v", 1000)],
+        [("t", 850), ("u", 850), ("v", 850)],
+    ]
+    for i, f in enumerate(fs):
+        assert len(f) == 3
+        assert f.metadata(("param", "level")) == ref[i]
+        assert f.to_fieldlist("numpy") is not f
+
+    # stream consumed, no data is available
+    assert sum([1 for _ in fs]) == 0
+
+
+@pytest.mark.parametrize(
+    "_kwargs",
+    [
+        {},
+        {"batch_size": 1},
+    ],
+)
+def test_grib_url_stream_single_batch(_kwargs):
+    ds = from_source(
+        "url",
+        earthkit_remote_test_data_file("examples/test6.grib"),
+        stream=True,
+        **_kwargs,
+    )
+
+    # no fieldlist methods are available
+    with pytest.raises(TypeError):
+        len(ds)
+
+    ref = [
+        ("t", 1000),
+        ("u", 1000),
+        ("v", 1000),
+        ("t", 850),
+        ("u", 850),
+        ("v", 850),
+    ]
+
+    for i, f in enumerate(ds):
+        assert f.metadata(("param", "level")) == ref[i], i
+
+    # stream consumed, no data is available
+    assert sum([1 for _ in ds]) == 0
+
+
+@pytest.mark.parametrize(
+    "_kwargs,expected_meta",
+    [
+        ({"batch_size": 3}, [["t", "u", "v"], ["t", "u", "v"]]),
+        ({"batch_size": 4}, [["t", "u", "v", "t"], ["u", "v"]]),
+    ],
+)
+def test_grib_url_stream_multi_batch(_kwargs, expected_meta):
+    ds = from_source(
+        "url",
+        earthkit_remote_test_data_file("examples/test6.grib"),
+        stream=True,
+        **_kwargs,
+    )
+
+    # no methods are available
+    with pytest.raises(TypeError):
+        len(ds)
+
+    for i, f in enumerate(ds):
+        assert len(f) == len(expected_meta[i])
+        f.metadata("param") == expected_meta[i]
+
+    # stream consumed, no data is available
+    assert sum([1 for _ in ds]) == 0
+
+
+def test_grib_url_stream_in_memory():
+    ds = from_source(
+        "url",
+        earthkit_remote_test_data_file("examples/test6.grib"),
+        stream=True,
+        batch_size=0,
+    )
+
+    assert len(ds) == 6
+
+    expected_shape = (6, 7, 12)
+    ref = ["t", "u", "v", "t", "u", "v"]
+    val = []
+
+    # iteration
+    for f in ds:
+        v = f.metadata("param")
+        val.append(v)
+
+    assert val == ref, "iteration"
+
+    # metadata
+    val = []
+    val = ds.metadata("param")
+    assert val == ref, "method"
+
+    # data
+    assert ds.to_numpy().shape == expected_shape
+
+    ref = np.array(
+        [
+            272.56417847,
+            -6.28688049,
+            7.83348083,
+            272.53916931,
+            -4.89837646,
+            8.66096497,
+        ]
+    )
+
+    vals = ds.to_numpy()[:, 0, 0]
+    assert np.allclose(vals, ref)
+
+
+def test_grib_save_when_loaded_from_url_stream():
+    ds = from_source(
+        "url",
+        earthkit_remote_test_data_file("examples/test6.grib"),
+        stream=True,
+        batch_size=0,
+    )
+    assert len(ds) == 6
+    with temp_file() as tmp:
+        ds.save(tmp)
+        ds_saved = from_source("file", tmp)
+        assert len(ds) == len(ds_saved)
+
+
+@pytest.mark.parametrize(
+    "_kwargs",
+    [
+        {},
+        {"batch_size": 1},
+    ],
+)
+def test_grib_multi_url_stream_single_batch(_kwargs):
+    ds = from_source(
+        "url",
+        [
+            earthkit_remote_test_data_file("examples/test.grib"),
+            earthkit_remote_test_data_file("examples/test4.grib"),
+        ],
+        stream=True,
+        **_kwargs,
+    )
+
+    # no fieldlist methods are available
+    with pytest.raises(TypeError):
+        len(ds)
+
+    ref = [
+        ("2t", 0),
+        ("msl", 0),
+        ("t", 500),
+        ("z", 500),
+        ("t", 850),
+        ("z", 850),
+    ]
+
+    for i, f in enumerate(ds):
+        assert f.metadata(("param", "level")) == ref[i], i
+
+    # stream consumed, no data is available
+    assert sum([1 for _ in ds]) == 0
+
+
+@pytest.mark.parametrize(
+    "_kwargs,expected_meta",
+    [
+        ({"batch_size": 2}, [["2t", "msl"], ["t", "z"], ["t", "z"]]),
+        ({"batch_size": 3}, [["2t", "msl", "t"], ["z", "t", "z"]]),
+        ({"batch_size": 4}, [["2t", "msl", "t", "z"], ["t", "z"]]),
+    ],
+)
+def test_grib_multi_url_stream_batch(_kwargs, expected_meta):
+    ds = from_source(
+        "url",
+        [
+            earthkit_remote_test_data_file("examples/test.grib"),
+            earthkit_remote_test_data_file("examples/test4.grib"),
+        ],
+        stream=True,
+        **_kwargs,
+    )
+
+    # no methods are available
+    with pytest.raises(TypeError):
+        len(ds)
+
+    for i, f in enumerate(ds):
+        assert len(f) == len(expected_meta[i])
+        f.metadata("param") == expected_meta[i]
+
+    # stream consumed, no data is available
+    assert sum([1 for _ in ds]) == 0
+
+
+def test_grib_multi_url_stream_memory():
+    ds = from_source(
+        "url",
+        [
+            earthkit_remote_test_data_file("examples/test.grib"),
+            earthkit_remote_test_data_file("examples/test4.grib"),
+        ],
+        stream=True,
+        batch_size=0,
+    )
+
+    assert len(ds) == 6
+
+    md_ref = [
+        ("2t", 0),
+        ("msl", 0),
+        ("t", 500),
+        ("z", 500),
+        ("t", 850),
+        ("z", 850),
+    ]
+
+    val = []
+
+    # iteration
+    for f in ds:
+        v = f.metadata(("param", "level"))
+        val.append(v)
+
+    assert val == md_ref, "iteration"
+
+    # metadata
+    val = []
+    val = ds.metadata(("param", "level"))
+    assert val == md_ref, "method"
+
+    # data
+    with pytest.raises(ValueError):
+        ds.to_numpy().shape
+
+    # first part
+    expected_shape = (2, 11, 19)
+    assert ds[0:2].to_numpy().shape == expected_shape
+
+    ref = np.array([262.78027344, 101947.8125])
+
+    vals = ds[0:2].to_numpy()[:, 0, 0]
+    assert np.allclose(vals, ref)
+
+    # second part
+    expected_shape = (4, 181, 360)
+    assert ds[2:].to_numpy().shape == expected_shape
+
+    ref = np.array([228.04600525, 48126.859375, 246.61032104, 11786.1132812])
+
+    vals = ds[2:].to_numpy()[:, 0, 0]
+    assert np.allclose(vals, ref)
+
+    # slicing
+    r = ds[0:3]
+    assert len(r) == 3
+    val = r.metadata(("param", "level"))
+    assert val == md_ref[0:3]
+
+    r = ds[-2:]
+    assert len(r) == 2
+    val = r.metadata(("param", "level"))
+    assert val == md_ref[-2:]
+
+    r = ds.sel(param="t")
+    assert len(r) == 2
+    val = r.metadata(("param", "level"))
+    assert val == [
+        ("t", 500),
+        ("t", 850),
+    ]
+
+
+if __name__ == "__main__":
+    from earthkit.data.testing import main
+
+    main()