iOS Secrets: Blending Images without CoreImage
By: Riccardo Cipolleschi
Tags: User Experience, Swift, Apple, iOS, Apps

User Interface

iOS Secrets: Blending Images without CoreImage

How to blend and compose your UIView without using CoreImage and by keeping their identities.

This week I was exploring some API to work with images and to apply some filters. The default way to do that is by using the CoreImage framework. It leverages the GPU to apply special filters to your images so that you can plug them into your UI with some amazing effects.

Everything seems to work fine if you have to work with a single image. The typical steps to apply a filter are the following:

Given a CIImage , we can use the applyingFilter(_:parameters:) method to obtain a new image, modified by the filter. Then, we can set the new image into our UIImageView to render it on the screen.

Every filter has its own set of parameters. The Apple API’s are not really type-safe. They are based on strings and moreover not all the filters use the same parameters. Here, you can find the official guide on all the filters, organized in categories.

When we work with filters that use a single input image, everything seems fine. If we have to move the original image, rotate or zoom it, we can perform these operations even if we have applied a filter on it. We can just move the filtered image and everything works and look fine.

CompositingFilters

If we go and look to the CICategoryCompositeOperation category, things get trickier. This category, in fact, contains filters that need two images to work properly: they need an input and a background image.

If we don’t need to perform further operations, we can apply the composition normally — in this case, the approach is actually the same as the previous one. We need to create two CIImage s, apply the filter by specifying the kCIInputBackgroundImageKey parameter by passing the second ciImage , and the output will be a third image that is the composition of the two:

However, what happens now if the user would like to move the foreground image or maybe they want to apply a rotation on it? The two images have been merged together by CoreImage . Yes, it is still doable with some tricks: we can keep the original images in memory, apply the transformation, and filtering them again.

However the question is: “Is there a simpler, more straightforward way?”

CALayer.compositingFilter

The answer to the previous question is, of course, yes.

CALayer has this wonderful property called compositingFilter . If we go and read the documentation, however, we can read a small little note, at the end of the doc:

Special Considerations This property is not supported on layers in iOS.

Oh no! That was unexpected. But… is it really true?

Indeed, if we try to copy and paste the documentation example in a playground, it does not work. It seems that the documentation is trustworthy in this. Maybe we need to go back to the hard way and get ready for some manual work… Or maybe not?

Hacking the CALayer

By digging on the web, I found this post on stack overflow that shows a way to use the above-mentioned property in iOS.

It looks like that the property works, but not in the way described by the documentation. The main point to keep in mind are:

We don’t need to create and set a CIFilter object.

object. Filters are not identified by the actual CIFilter names. This property uses a slightly different approach to configure the filters.

By experimenting with them, I discovered some rules to make it works:

Compositing filters come in 2 flavors: BlendMode filters and Compositing filters. To apply a BlendMode filter, we need to set a value that is a String , it does not have the initial CI , and it has the first letter (after the removal of CI ) lowercased. For example, to apply a CIScreenBlendMode we need to set the layer.compositingFilter property to screenBlendMode . To apply a Compositing filter, we need to set a value that is a String , it does not have the initial CI , it has the first letter (after the removal of CI ) lowercased, and it does not have the final Compositing word. For example, to apply a CIMaximumCompositing filter we just need to set the layer.compositingFilter property to maximum .

Demo

To show how this works, I prepared a simple app. The app has two tabs, a BlendMode tab and a Compositing tab. It is possible to download the code from this public repo.

Blend Mode

In the BlendMode tab, an image is used as a background and then we render an image for every BlendMode filter on top of it.

The final effect is the one shown in this video.

Thanks to this video, it is possible to see all the available blend modes and how they work. By clicking on a filter, it is possible to see which blend mode is applied between the tapped image and the background.

The relevant details to achieve this result are highlighted in the following snippet.

Note: I’ll report the entity the snippet belongs to but I’ll omit the complete implementation. It is possible to download the full code from Github.

In the first part of the snippet, there is a ViewModel (VM) that contains all the information for the main view. When it is initialized, we query the system to retrieve all the available CIFilter s. We filter out what is not a BlendMode , we drop the CI prefix, and we lowercase the initial letter.

Then, in the View , we make sure to set the collectionView.backgroundView property to the same UIView we want to use as a kCIInputBackgroundImageKey for our filters.

In the collectionView(_:cellForItemAt:) we create a VM for the cell where we pass the filter that needs to be used to blend the cell with the background. The filter is finally applied in the BlendModeCell ’s update(oldModel:) method, by retrieving it from the model.

Compositing

The second tab is the Compositing one. In this one, we pick two different images and we see how they can be composed together with the system filters. These two images are laid out in the top half of the screen, overlapping a bit so that we can see their composition.

In the lower half of the view, there are all the available filters: by tapping on one of them, we apply it to the images in the upper half. Here is the video for this second feature:

As for the BlendMode tab, the most relevant parts of the code are summarized in the following snippet:

We first set up the VM by fetching all the Compositing filters and removing the CI prefix and the Compositing suffix. Finally, we create the filter name by lowercasing the first letter.

In the View , we simply add both the images to the view as we usually do. In the update(oldModel:) method, we set the images and we set the layer.compositingFilter on the topmost image: in this way, UIKit will blend that image with the one immediately behind it.

To make sure that the composition happens, we overlap the background image with the topmost one in the layout. Both images have a width of 75% of the width view. The background image starts from the left, while the topmost image starts from the right. The net result is that the two images are overlapped for half of the screen width.

Conclusion

In this article, we explored how to blend and compose images in UIKit without using CoreImage explicitly. This allows us to keep the original UIImageView s as if they were not blended.

This result can be achieved by using an API that was created for macOS but that works beautifully well also on iOS. Notice that the CALayer has a couple more filter properties ( filters and backgroundFilters ) that, unfortunately, does not work as well. Actually, they don’t work at all on iOS.

But, at least, we can blend and compose images easily, creating magical effects in our iOS applications.