Repository: vega/altair

Issue:
Add example showing how to render numpy image arrays as tooltip images
I think it could be helpful to show how images that are represented as numpy arrays can be rendered in tooltips in altair. I can add a doc example of this. Maybe in [the tutorials/case studies](https://altair-viz.github.io/case_studies/exploring-weather.html) section? We only have one example there currently. Another option would be to combine it with https://altair-viz.github.io/gallery/image_tooltip.html and create a new page in the user guide on images, but that 's more involved.

Here is the code and a video of the output. Note that this would add scipy as a documentation dependency (I could probably get around that, but I have another example I want to add that requires scipy so thought I might as well ask now if it is ok to add it). The images are not too large, the size of the chart saved as an html file is around 200kb.

1. Create some example image arrays with blobs in them and measure the area of the blobs.
    ```python
    import numpy as np
    import pandas as pd
    from scipy import ndimage as ndi
    
    rng = np.random.default_rng([ord(c) for c in 'altair'])
    n_rows = 200
    
    def create_blobs(img_width=96, n_dim=2, thresh=0.0001, sigmas=[0.1, 0.2, 0.3]):
        """Helper function to create blobs in the images"""
        shape = tuple([img_width] * n_dim)
        mask = np.zeros(shape)
        points = (img_width * rng.random(n_dim)).astype(int)
        mask[tuple(indices for indices in points)] = 1
        return ndi.gaussian_filter(mask, sigma=rng.choice(sigmas) * img_width) > thresh
    
    df = pd.DataFrame({
        'img1': [create_blobs() for _ in range(n_rows)],
        'img2': [create_blobs(sigmas=[0.15, 0.25, 0.35]) for _ in range(n_rows)],
        'group': rng.choice(['a', 'b', 'c'], size=n_rows)
    })
    df[['img1_area', 'img2_area']] = df[['img1', 'img2']].applymap(np.mean)
    df
    ```

2. Convert the numpy arrays to base64 encoded strings that will show in the tooltip

    ```python
    from io import BytesIO
    from PIL import Image, ImageDraw
    import base64
    
    
    def create_tooltip_image(df_row):
        # Concatenate images to show together in the tooltip
        img_gap = np.ones([df_row['img1'].shape[0], 10])  # 10 px white gap between imgs
        img = Image.fromarray(
            np.concatenate(
                [
                    df_row['img1'] * 128,  # grey
                    img_gap * 255,  # white
                    df_row['img2'] * 128
                ],
                axis=1
            ).astype('uint8')
        )
        
        # Optional: Burn in labels as pixels in the images
        ImageDraw.Draw(img).text((3, 0), 'img1', fill=255)
        ImageDraw.Draw(img).text((3 + df_row['img1'].shape[1] + img_gap.shape[1], 0), 'img2', fill=255)
        
        # Convert to base64 encoded image string that can be displayed in the tooltip
        buffered = BytesIO()
        img.save(buffered, format="PNG")
        img_str = base64.b64encode(buffered.getvalue()).decode()
        return f"data:image/png;base64,{img_str}"
    
    # The column with the image must be called "image" in order for it to trigger the image rendering in the tooltip
    df['image'] = df[['img1', 'img2']].apply(create_tooltip_image, axis=1)
    
    # Dropping the images since they are large an no longer needed
    df = df.drop(columns=['img1', 'img2'])
    df
    ```
3. Create a chart to show the images


    ```python
    import altair as alt
    
    alt.Chart(df, title='Area of grey blobs').mark_circle().encode(
        x='group',
        y=alt.Y(alt.repeat(), type='quantitative'),
        tooltip=['image'],
        color='group'
    ).repeat(
        ['img1_area', 'img2_area']
    )
    ```


https://github.com/altair-viz/altair/assets/4560057/45ccc43f-c8a4-4b3b-bb42-ed0b18cd9703




Discussion:
I was not aware that this is possible. That's super cool! :) I wouldn't mind having scipy as a new doc dependency and think it would be great to have this in the example gallery.

Resolving patch:
diff --git a/sphinxext/altairgallery.py b/sphinxext/altairgallery.py
--- a/sphinxext/altairgallery.py
+++ b/sphinxext/altairgallery.py
@@ -86,7 +86,7 @@
    :hidden:
 
    Gallery <self>
-   Tutorials <../case_studies/exploring-weather>
+   Tutorials <../case_studies/index>
 """
 )
 


Test patch: