How to use LiipImagineBundle to manage thumbnails through Amazon S3
Tags: experience, amazonwebservices, php, symfony
Published: 2024-03-16 10:07:54

---
title: How to use LiipImagineBundle to manage thumbnails through Amazon S3
published: true
date: 2016-01-25 16:45:35 UTC
tags: Experience,AmazonWebServices,PHP,Symfony
canonical_url: https://io.serendipityhq.com/experience/how-to-use-liipimaginebundle-to-manage-thumbnails-through-amazon-s3/
---

Configure `LiipImagineBundle` to create thumbnails of images stored on Amazon S3 and save a cache version of them again on S3 may be really painful.

In this post I’ll try to guide you step by step in the configuration process, to have a full configured data flow to create thumbnails stored on Amazon S3 using `LiipImagineBundle`.

## Which is our goal

Our goal is to configure the following data flow:

1. Get a source image file from AWS S3 (from the `[bucket_name]/images` folder);
2. Manipulate it in some ways with `LiipImagineFilters` (for example, the [`thumbnail`](https://github.com/liip/LiipImagineBundle/blob/master/Resources/doc/filters.rst#the-thumbnail-filter) filter);
3. Save the manipulated image again to AWS S3 (into the `bucket_name/cache` folder).

As the configuration of the full data flow could be difficult and prone to errors, we will use a simple configuration to reduce to minimum possibilities of issues.

So we will try to get an already uploaded image from the `images` folder in our bucket on S3 (hardcoding the path in the Twig template), resize it applying custom filters `test_medium_thumb` and `test_small_thumb` (that we will configure), then we will save again the resized image in the `cache` folder into same bucket on S3.

Before starting with the real configuration of the `LiipImagineBundle` we have to prepare our test bucket and our test `route` to simplify things.

We will, in fact, create a dedicate `action` in a `controller` to test in the tiniest way our filters. This will help us to test in a simpler manner the funcionalities of `LiipImagineBundle`.

## Install the required bundles: `LiipImagineBundle` and `KnpGaufretteBundle`

To make all things work we need to install and activate, besides `LiipImagineBundle`, also `KnpGaufretteBundle`.

So, [install and activate `LiipImagineBundle`](https://github.com/liip/LiipImagineBundle/blob/master/Resources/doc/installation.rst) and [install and activate `KnpGaufretteBundle`](https://github.com/KnpLabs/KnpGaufretteBundle#installation).

> [Gaufrette](https://github.com/KnpLabs/Gaufrette) is a PHP5 library that provides a filesystem abstraction layer.

Gaufrette is required to make `LiipImagineBundle` to get and save files from and to Amazon S3 buckets.

For the moment believe me and install the bundles: continuing in reading of this post you’ll better understand why we need also `KnpGaufretteBundle`.

## Prepare our app for the test

Now that we have installed and activated our required bundles, lets prepare our app to test the data flow:

1. Create in your bucket the folder `images`;
2. Into this `images` folder upload a big image (I used [this](https://www.symfony.fi/files/2015-05/symfony2.png));
3. Create a `cache` folder in your bucket;
4. Create a `DefaultController::TestThumbAction` in your `DefaultController` (or in another controller: this action will be used to just render the test image and its thumbnail) that return the path of the just MANUALLY uploaded test image:
```
    /**
     * @Route("/test_thumb", name="thumb")
     * @Template()
     */
    public function testThumbAction()
    {
        $image = 'https://s3-'.
            $this->getParameter('amazon.s3.region').
            '.amazonaws.com/'.
            $this->getParameter('amazon.s3.bucket').
            '/images/symfony2.png';
        
        return [
            'image' => $image
        ];
    }
```
5. Create the template for the `DefaultController::TestThumbnailAction`:
```
<h1>Test resized with <code>test_small_thumb</code></h1>
{# NOTE THAT FOR THE MOMENT THE THUMBNAIL IS COMMENTED #}
{# <img src="{{ image | imagine_filter('test_small_thumb') }}" alt="" /> #}

<h1>Test resized with <code>test_medium_thumb</code></h1>
{# NOTE THAT FOR THE MOMENT THE THUMBNAIL IS COMMENTED #}
{# <img src="{{ image | imagine_filter('test_medium_thumb') }}" alt="" /> #}

<h1>Original Image</h1>
<img src="{{ image }}" alt="" />
```

Now go to `http://127.0.0.1:8000/test_thumb` and check that the test image is loaded correctly.

If the test image is not loaded, may be you have to set it as `public` on AWS S3 (select the image and in the `Actions` dropdown select `Make Public`).

Once you can see the test image in its original size, remove comments from the image with `test_small_thumb` filter applied (only from this for the moment!) and reload the page.

You’ll see this `Twig_Error_Runtime` exception

> An exception has been thrown during the rendering of a template (“Could not find configuration for a filter: test\_small\_thumb”) in <abbr title="/Users/Aerendir/Documents/JooServer/_Projects/Coommercio/Apps/TrustBack.Me/src">src</abbr>/AppBundle/Resources/views/Default/testThumb.html.twig at line 2.  
> **500** Internal Server Error – <abbr title="Twig_Error_Runtime">Twig_Error_Runtime</abbr>  
> **1** linked Exception: <abbr title="Liip\ImagineBundle\Exception\Imagine\Filter\NonExistingFilterException">NonExistingFilterException</abbr> »

Ok, we are now ready to start configuring `LiipImagineBundle` to create our thumbnails of images stored on Amazon S3 and save the cached thumbnail again on Amazon S3. Lets start!

## Understand the dataflow of `LiipImagineBundle`

The dataflow followed by `LiipImagineBundle` is described in the [documentation](https://github.com/liip/LiipImagineBundle/blob/master/Resources/doc/introduction.rst#basic-data-flow). I suggest you to read it to fully understand what we need to make the thumbnails being generated and saved to S3.

The short story is this:

1. STEP 1: `LiipImagineBundle` [retrieve the original image](https://github.com/liip/LiipImagineBundle/blob/master/Resources/doc/introduction.rst#retrieving-the-original-image) from S3 using a `DataLoader`.  
Our `DataLoader` is the [`StreamLoader`](https://github.com/liip/LiipImagineBundle/blob/master/Resources/doc/data-loader/stream.rst) (that uses Gaufrette to work).
2. STEP 2: The source image is then [processed](https://github.com/liip/LiipImagineBundle/blob/master/Resources/doc/introduction.rst#apply-filters-on-the-original-image) with [filters](https://github.com/liip/LiipImagineBundle/blob/master/Resources/doc/filters.rst).  
The filters we will configure are `test_medium_thumb` and `test_small_thumb`.
3. STEP 3: The resulting image file is then [saved (cached)](https://github.com/liip/LiipImagineBundle/blob/master/Resources/doc/introduction.rst#cache-the-filtered-image) to S3 using a [`CacheResolver`](https://github.com/liip/LiipImagineBundle/blob/master/Resources/doc/cache-resolvers.rst).  
The `CacheResolver` we’ll use is the `AwsS3Resolver.`

## STEP 1: Get the image source file from AWS S3 with Gaufrette

So, as told, the first step is to get the source image from the `images` folder in our bucket on AWS S3.

To do this, `LiipImagnineBundle` requires a [`DataLoader`](https://github.com/liip/LiipImagineBundle/blob/master/Resources/doc/data-loaders.rst).

`LiipImagnineBundle` comes with some bundled [`DataLoader`s](https://github.com/liip/LiipImagineBundle/tree/master/Resources/doc/data-loader). The default `DataLoader` is the [`FilesystemLoader`](https://github.com/liip/LiipImagineBundle/blob/master/Resources/doc/data-loader/filesystem.rst) as described also in the [Configuration chapter](https://github.com/liip/LiipImagineBundle/blob/master/Resources/doc/configuration.rst) of the documentation.

To load images from Amazon S3, we need another `DataLoader`, the [`StreamLoader`](https://github.com/liip/LiipImagineBundle/blob/master/Resources/doc/data-loader/stream.rst):

> The `Liip\ImagineBundle\Binary\Loader\StreamLoader` allows to read images from any stream (http, ftp, and others…) registered thus allowing you to serve your images from literally anywhere.
> 
> <footer><cite title="LiipImagineBundle Documentation, Stream Loade">LiipImagineBundle Documentation, Stream Loader</cite></footer>

The easier way to use this loader is configuring it to use [Gaufrette](https://github.com/KnpLabs/Gaufrette).

So we have to first configure Gaufrette and then use it with the `LiipImagineBundle`‘`StreamLoader`.

### Understand how Gaufrette works

At this point Gaufrette should be already installed as you should have already installed and activated `KnpGaufretteBundle`.

But, before continuing, just a little bit of theory to understand how Gaufrette works.

As told before,

> Gaufrette is a PHP 5.3+ library providing a filesystem abstraction layer. This abstraction layer allows you to develop applications without needing to know where all their media files will be stored or how.

In other words, you interact with Gaufrette’s API that emulates actions you can take on a filesystem to manage files, so, when you decide to switch from the local filesystem to a cloud environment, you have to simply change some parameters in the configuration. Clean, simple and easy!

If you like, read [some more details](http://knplabs.com/en/blog/give-your-projects-a-gaufrette) on the Knp Labs’ blog.

Anyway, to make your application “storage agnostic”, Gaufrette uses two main concepts: `Adapters` and `Filesystems`.

Your application will ever communicate with a `Filesystem` object. An `Filesystem` object uses a `Adapter` to communicate with the “storage” and perform the real actions on it.

[Here the details](https://github.com/KnpLabs/Gaufrette#setup-your-filesystem) about this simple process.

### Introducing Gaufrette `StreamWrapper`

To make `LiipImagineBundle` `StreamLoader` work with images we need to understand another concept used by Gaufrette, the last one: `StreamWrapper`.

As we don’t simply want to interact with the filesystem, but we want also to manipulate images, we need a [`StreamWrapper`](https://github.com/KnpLabs/Gaufrette#streaming-files) that we’ll then use as `DataLoader` in `LiipImagineBundle`.

A `StreamWrapper` permits to [register a new stream wrapper](http://php.net/manual/en/function.stream-wrapper-register.php) so we’ll can get the contents of the image and use them to create a new image.

### Configuring `KnpGaufretteBundle` to work with AWS S3

So, after the theory, finally the practice! Lets configure `KnpGaufretteBundle`.

So, we need to configure those three things:

1. An `Adapter`: we’ll use the [`AwsS3Adapter`](https://github.com/KnpLabs/KnpGaufretteBundle#awss3)
2. A `Filesystem` that uses the `AwsS3Adapter`: we’ll call it `filesystem_aws_S3_images`
3. A `StreamWrapper` (we don’t have to do anything more than writing a directive in our `config.yml` file to configure it)

#### Configure the AwsS3Adapter for `KnpGaufretteBundle`

The `AwsS3Adapter` [needs the credentials](https://github.com/KnpLabs/Gaufrette#using-amazon-s3) to access the bucket on Amazon AWS S3, so, we first configure these credentials, then we’ll configure the `S3Client` (that uses the credentials):

```
shq.amazon.s3Credentials:
    class: Aws\Credentials\Credentials
    arguments: ["%amazon.s3.key%", "%amazon.s3.secret%"]

shq.amazon.s3:
    class: Aws\S3\S3Client
    arguments:
        - version: %amazon.s3.version%
          region: %amazon.s3.region%
          credentials: "@shq.amazon.s3Credentials"
```

Now, we can configure the `AwsS3Adapter`, the `Filesystem` and the `StreamWrapper`:

```
# config.yml
# KnpGaufretteBundle Configuration
knp_gaufrette:
    adapters:
        adapter_aws_s3_images:
            aws_s3:
                service_id: "shq.amazon.s3" # ! ! ! Without @ ! ! !
                bucket_name: "%amazon.s3.bucket%"
                options:
                    directory: 'images'
                    create: true
    filesystems:
        filesystem_aws_s3_images:
            adapter: adapter_aws_s3_images
    stream_wrapper: ~
```

As told in the [documentation](https://github.com/KnpLabs/KnpGaufretteBundle#stream-wrapper),

> The `stream_wrapper` settings allow you to register filesystems with a specified domain and then use as a stream wrapper anywhere in your code like: `gaufrette://domain/file.txt`
> 
> <footer><cite title="GaufretteBundle Documentation, Stream Wrappers">GaufretteBundle Documentation, Stream Wrappers</cite></footer>

#### Testing that `filesystem_aws_s3_images` works

Now that we have configured our `Adapter` and our `Filesystem`, we should test that it really works as expected.

Once configured, `Filesystem`s are accessibile from the container, so your `DefaultController::testThumbAction()` write the following code:

```
    /**
     * @Route("/test_thumb", name="thumb")
     * @Template()
     */
    public function testThumbAction()
    {
        $filesystem = $this->get('knp_gaufrette.filesystem_map')->get('filesystem_aws_s3_images');
        $file = $filesystem->get('symfony2.png');

        /*
         * Note the use of the dump() function.
         * If you don't have the VarDumperComponent installed, use var_dump().
         * @see http://symfony.com/doc/current/components/var_dumper/introduction.html
         */
        dump($file);die;
        
        $image = 'https://s3-'.

        ...
    }

```

If the `dump()` shows you a `Gaufrette\File` object, then… CONGRATULATIONS! Gaufrette is correctly configured! ![🙂](https://s.w.org/images/core/emoji/14.0.0/72x72/1f642.png)

As we know that Gaufrette works, we can now use it with `LiipImgineBundle`!

#### Using Gaufrette as `DataLoader` for `LiipImagineBundle`

As told, we need a `DataLoader` to make `LiipImagineBundle` get the real content of the file so it can create a thumbnails.

Now that we have a working Gaufrette, we need to use it to configure the [`StreamLoader`](https://github.com/liip/LiipImagineBundle/blob/master/Resources/doc/data-loader/stream.rst#streamloader).

We have to alternatives to configure a `DataLoader`: using the factory and using a defined service.  
We’ll use the factory method, so we have all configuration in one place (I find this more comfortable, but you can use the method you like).

So, to define the `DataLoader` using the [factory method](https://github.com/liip/LiipImagineBundle/blob/master/Resources/doc/data-loader/stream.rst#using-factory), put this in your `config.yml` file:

```
liip_imagine:
    loaders:
        loader_aws_s3_images:
            stream:
                # This refers to knp_gaufrette filesystems configuration
                wrapper: gaufrette://filesystem_aws_s3_images/
```

Perfect: we have done. Now we have to configure custom filters.

## STEP 2: Define filters to resize images and create the thmbnails

Now that we can get images from AWS S3, it’s time to define our custom filters to manipulate them.

We will create two filters: `test_medium_thumb` and `test_small_thumb`.

To define filters we have to edit the `config.yml` file:

```
liip_imagine:
    loaders:
...
    filter_sets:
        test_medium_thumb:
            data_loader: loader_aws_s3_images
            # We don't yet have a cache resolver configured
            cache: cache_resolver_aws_s3
            quality: 75
            filters:
                thumbnail: { size: [400, 400], mode: outbound }
        test_small_thumb:
            data_loader: loader_aws_s3_images
            # We don't yet have a cache resolver configured
            cache: cache_resolver_aws_s3
            quality: 75
            filters:
                thumbnail: { size: [100, 100], mode: outbound }

```

Done! Really simple, doesn’t it?

But, as you can see, we have the directive `cache` in each of the two defined filters: these directives use the `CacheResolver` called `cache_resolver_aws_s3`.

So, as it may be yet clear, we have to define the `cache_resolver_aws_s3`. We are fastly moving toward the STEP 3! ![🙂](https://s.w.org/images/core/emoji/14.0.0/72x72/1f642.png)

## STEP 3: Define the `CacheResolver` `AwsS3Resolver`

As we want to save our generated thumbnails to AWS S£, we need to use the `CacheResolver` [`AwsS3Resolver`](https://github.com/liip/LiipImagineBundle/blob/master/Resources/doc/cache-resolver/aws_s3.rst).

In this case, too, we can decide to define our resolver as a service or using the factory.  
In this case, too, I’ll use the factory, but you can choose to use the method you like.

So, in the `config.yml` file:

```
liip_imagine:
    loaders:
        ...
    resolvers:
       cache_resolver_aws_s3:
          aws_s3:
              client_config:
                  credentials:
                      key: %amazon.s3.key%
                      secret: %amazon.s3.secret%
                  region: %amazon.s3.region%
                  version: %amazon.s3.version%
              get_options:
                  Scheme: 'https'
              put_options:
                  CacheControl: 'max-age=86400'
    filter_sets:
        ...
```

## Test that all works well

And now that we have all configured, it’s time to test!

### Edit `DefaultController::TestThumbAction`

First: change the code into the action:

```
    /**
     * @Route("/test_thumb", name="thumb")
     * @Template()
     */
    public function testThumbAction()
    {
        //$filesystem = $this->get('knp_gaufrette.filesystem_map')->get('filesystem_aws_s3_images');
        //$file = $filesystem->get('symfony2.png');

        /*
         * Note the use of the dump() function.
         * If you don't have the VarDumperComponent installed, use var_dump().
         * @see http://symfony.com/doc/current/components/var_dumper/introduction.html
         */
        //dump($file);die;

        /*$image = 'https://s3-'.
            $this->getParameter('amazon.s3.region').
            '.amazonaws.com/'.
            $this->getParameter('amazon.s3.bucket').
            '/images/symfony2.png';*/

        $image = 'symfony2.png';

        return [
            'image' => $image
        ];
    }
```

### Activate filters in the template

Now activate the filters in the template removing comments:

```
<h1>Test resized with <code>test_small_thumb</code></h1>
<img src="{{ image | imagine_filter('test_small_thumb') }}" alt="" />

<h1>Test resized with <code>test_medium_thumb</code></h1>
<img src="{{ image | imagine_filter('test_medium_thumb') }}" alt="" />

<h1>Original Image</h1>
<img src="{{ image }}" alt="" />
```

### Test!

Reload your page at `http://127.0.0.1:8000/test_thumb` and, if you’ve done well all the steps, you’ll see something like this:

![590-screenshot-liip_imagine_bundle](https://io.serendipityhq.com/wp-content/uploads/sites/2/2016/01/590-screenshot-liip_imagine_bundle.png)

Obviously, “Original image” is no longer visible as the `path` isn’t correct (we commented it in the controller!).

Done: now you have `LiipImagineBundle` up and running. You can now save your images to Amazon AWS S3, get them and resize to thumbnail sizes and save them again on AWS S3! ![🙂](https://s.w.org/images/core/emoji/14.0.0/72x72/1f642.png)

## Some things to note: what happened behind the scenes

The first thing you should note is the URLs of the thumbnails.

The first time you load the page the resulting URLs are those:

```
<h1>Test resized with <code>test_small_thumb</code></h1>
<img src="http://127.0.0.1:8000/media/cache/resolve/test_small_thumb/symfony2.png" alt="" />

<h1>Test resized with <code>test_medium_thumb</code></h1>
<img src="http://127.0.0.1:8000/media/cache/resolve/test_medium_thumb/symfony2.png" alt="" />

<h1>Original Image</h1>
<img src="symfony2.png" alt="" />
```

As you can note, they are in this form: `media/cache/resolve/[filter_name]/`.

Try to reload the page.

The resulting URLs, this time, are those:

```
<h1>Test resized with <code>test_small_thumb</code></h1>
<img src="https://s3-eu-west-1.amazonaws.com/trustback-me-dev/test_small_thumb/symfony2.png" alt="" />

<h1>Test resized with <code>test_medium_thumb</code></h1>
<img src="https://s3-eu-west-1.amazonaws.com/trustback-me-dev/test_medium_thumb/symfony2.png" alt="" />

<h1>Original Image</h1>
<img src="symfony2.png" alt="" />
```

They refers to the public URL on AWS S3: `https://s3-eu-west-1.amazonaws.com/[bucket_name]/[filter_name/`.

In fact, the first time you load the images, if `LiipImagineBundle` cannot find a cached version, points to a controller defined in [`vendor/liip/imagine-bundle/Resources/config/routing.xml`](https://github.com/liip/LiipImagineBundle/blob/master/Resources/config/routing.xml)

The URL you see the first time points to the controller that applies the filters. Once the resulting image is cached, the next time `LiipImagineBundle` uses directly the URL of the bucket on AWS S3.

The second thing you should note, is that `LiipImagineBundle` automatically creates a folder for each filter in your bucket:

![590-screenshot-liip_imagine_bundle-bucket-details](https://io.serendipityhq.com/wp-content/uploads/sites/2/2016/01/590-screenshot-liip_imagine_bundle-bucket-details.png)

In practice, for each configured filter, the resulting images are saved into a dedicated folder.

## Troubleshooting

And finally, some advices to debug in case of issues:

1. Keep attention to differences between UPPERCASE and lowercase letters: maybe you can write `filesystem_aws_s3_images` in one place and `filesystem_aws_S3_images` in another place. Spot the difference!
2. As the bundle is usually usedfrom inside a template, you’ll don’t get exceptions or errors. If something doesn’t work, go to the logs! There there are all the errors: find them looking for strings like “gaufrette” or “liip\_imagine” or “liip/imagine-bundle” or “request.ERROR” or “php.DEBUG” and you’ll find all the info you need to understand what went wrong.

And finally, before to close this long post, remember that in the [Issues section](https://github.com/liip/LiipImagineBundle/issues/) on GitHub there are a lot of useful information to better understand how to use the bundle.

Here are a couple of interesting issues:

- [Complete S3 Setup: Original Images and cached Images on AWS S3](https://github.com/liip/LiipImagineBundle/issues/203)
- [need https option for aws stream](https://github.com/liip/LiipImagineBundle/issues/335)

Remember to “Make. Ideas. Happen.”.

I wish you flocking users, see you soon!

L'articolo [How to use LiipImagineBundle to manage thumbnails through Amazon S3](https://io.serendipityhq.com/experience/how-to-use-liipimaginebundle-to-manage-thumbnails-through-amazon-s3/) proviene da [ÐΞV Experiences by Serendipity HQ](https://io.serendipityhq.com).