Repository: matplotlib/matplotlib

Issue:
[Bug]: inconsistent colorbar pad for `ImageGrid` with `cbar_mode="single"`
### Bug summary

`ImageGrid` with `cbar_mode="single"` adds the `axes_pad` between the axes and the colorbar if the colorbar is to the left or at the bottom. That seems inconsistent and unnecessary as the space can be controlled with `cbar_pad`.

### Code for reproduction

```Python
import matplotlib.pyplot as plt
from mpl_toolkits.axes_grid1 import ImageGrid


fig = plt.figure(figsize=(6, 6))

subfigs = fig.subfigures(2, 2, wspace=0.07, hspace=0.1, linewidth=1)
figs = subfigs.flatten()

locations = ["left", "bottom", "right", "top"]

for fig, location in zip(figs, locations):

    fig.set_edgecolor("r")

    axgr = ImageGrid(
        fig,
        111,
        nrows_ncols=(1, 1),
        axes_pad=1. / 2.54,
        cbar_mode="single",
        cbar_location=location,
        cbar_pad=0. / 2.54,
    )
    ax = axgr.axes_all[0]
    ax.text(0.5, 0.5, f"{location=}", transform=ax.transAxes, va="center", ha="center")
    ax.set(xticks=[], yticks=[])
    
    cbax = axgr.cbar_axes[0]
    cbax.set(xticks=[], yticks=[])

    plt.savefig("cbar_mode_single.png")
```


### Actual outcome

![cbar_mode_single](https://github.com/matplotlib/matplotlib/assets/10194086/526d6c03-5228-485e-83c6-a9781d72a90e)


### Expected outcome

I expect all colorbars to hug the axes.

### Additional information

- As mentioned this only happens for `cbar_mode="single"` and `cbar_location` `"left"` or `"bottom"`
- This behavior seems to have been around since forever.
- There is the `cbar_pad` param which can be used to control the distance between axes and colorbar
- Note that both `cbar_pad` and `axes_pad` are added between the axes and the colorbar
- I am happy to open to open a PR, but don't see a way to deprecate this gracefully

---

- the space is added in these lines respectively:
   https://github.com/matplotlib/matplotlib/blob/479bd7a0cad2b77ef522c1dd6d65080a4da62b4f/lib/mpl_toolkits/axes_grid1/axes_grid.py#L442-L443
   https://github.com/matplotlib/matplotlib/blob/479bd7a0cad2b77ef522c1dd6d65080a4da62b4f/lib/mpl_toolkits/axes_grid1/axes_grid.py#L471-L472
- This is tested in
https://github.com/matplotlib/matplotlib/blob/479bd7a0cad2b77ef522c1dd6d65080a4da62b4f/lib/mpl_toolkits/axes_grid1/tests/test_axes_grid1.py#L420
- and the image: https://github.com/matplotlib/matplotlib/blob/479bd7a0cad2b77ef522c1dd6d65080a4da62b4f/lib/mpl_toolkits/axes_grid1/tests/baseline_images/test_axes_grid1/image_grid_single_bottom_label_mode_1.png

### Operating system

_No response_

### Matplotlib Version

3.8.4

### Matplotlib Backend

_No response_

### Python version

_No response_

### Jupyter version

_No response_

### Installation

conda


Discussion:
This seems like a clear bug with no obvious backcompat route; I would suggest changing to not including axes_pad when positioning the colorbar and having a clear note in the API changelog.



Patch:
diff --git a/doc/users/next_whats_new/mpl_toolkit_colorbar_pad.rst b/doc/users/next_whats_new/mpl_toolkit_colorbar_pad.rst
new file mode 100644
index 000000000000..f22b7c79089c
--- /dev/null
+++ b/doc/users/next_whats_new/mpl_toolkit_colorbar_pad.rst
@@ -0,0 +1,6 @@
+Fix padding of single colorbar for ``ImageGrid``
+------------------------------------------------
+
+``ImageGrid`` with ``cbar_mode="single"`` no longer adds the ``axes_pad`` between the
+axes and the colorbar for ``cbar_location`` "left" and "bottom". If desired, add additional spacing
+using ``cbar_pad``.
diff --git a/lib/mpl_toolkits/axes_grid1/axes_grid.py b/lib/mpl_toolkits/axes_grid1/axes_grid.py
index 315a7bccd668..63888b1932ff 100644
--- a/lib/mpl_toolkits/axes_grid1/axes_grid.py
+++ b/lib/mpl_toolkits/axes_grid1/axes_grid.py
@@ -358,6 +358,11 @@ def __init__(self, fig,
         cbar_location : {"left", "right", "bottom", "top"}, default: "right"
         cbar_pad : float, default: None
             Padding between the image axes and the colorbar axes.
+
+            .. versionchanged:: 3.10
+                ``cbar_mode="single"`` no longer adds *axes_pad* between the axes
+                and the colorbar if the *cbar_location* is "left" or "bottom".
+
         cbar_size : size specification (see `.Size.from_any`), default: "5%"
             Colorbar size.
         cbar_set_cax : bool, default: True
@@ -439,7 +444,7 @@ def _init_locators(self):
             self.cbar_axes[0].set_visible(True)
 
         for col, ax in enumerate(self.axes_row[0]):
-            if h:
+            if col != 0:
                 h.append(self._horiz_pad_size)
 
             if ax:
@@ -468,7 +473,7 @@ def _init_locators(self):
         v_ax_pos = []
         v_cb_pos = []
         for row, ax in enumerate(self.axes_column[0][::-1]):
-            if v:
+            if row != 0:
                 v.append(self._vert_pad_size)
 
             if ax:


Test patch:
diff --git a/lib/mpl_toolkits/axes_grid1/tests/test_axes_grid1.py b/lib/mpl_toolkits/axes_grid1/tests/test_axes_grid1.py
index 7c444f6ae178..d5a79a21c000 100644
--- a/lib/mpl_toolkits/axes_grid1/tests/test_axes_grid1.py
+++ b/lib/mpl_toolkits/axes_grid1/tests/test_axes_grid1.py
@@ -424,7 +424,7 @@ def test_image_grid_single_bottom():
 
     fig = plt.figure(1, (2.5, 1.5))
     grid = ImageGrid(fig, (0, 0, 1, 1), nrows_ncols=(1, 3),
-                     axes_pad=(0.2, 0.15), cbar_mode="single",
+                     axes_pad=(0.2, 0.15), cbar_mode="single", cbar_pad=0.3,
                      cbar_location="bottom", cbar_size="10%", label_mode="1")
     # 4-tuple rect => Divider, isinstance will give True for SubplotDivider
     assert type(grid.get_divider()) is Divider