Creating Instagram and Facebook AR Filters with Spark AR Studio (Part 3) — The Patch Editor
By: Harsh Chandra
Tags: Instagram, Spark Ar, Social Ar, Heartbeat, Augmented Reality

Step 7:

You will notice that the rectangles are not moving with the head in the simulator even though the Canvas and the Rectangles objects are present inside the Face Tracker.

This is happening because the Canvas is pinned to ‘Camera Space’ instead of ‘World Space.’ It simply means that the Canvas object is present in 2D screen space instead of 3D world space. You can change this easily by selecting the Canvas object and changing its ‘mode’ property to ‘World Space’ in the properties panel on the right.

Now the rectangles are probably moving along with the head however they are covering the user’s face.

To fix this, you need to keep the Canvas object selected and drag the green up arrow upwards to a desired height in the central viewport area. This may seem difficult to do while the simulated user keeps moving. To make it easier, just pause the simulator using the pause button on the far left.

Step 8:

Next, we need to add a Material to both the Rectangle objects. Select the ‘Title’ rectangle and click on the ‘+’ icon on the right panel under the Materials heading. This will add the default material to the ‘Title’ rectangle.

Now select the material you just created and change its ‘Shader Type’ property to ‘flat.’ We chose ‘flat’ because we are going to add an image to this rectangle and keep its original colors instead of lighting it up with artificial lighting. You can learn more about shader types here.

Repeat the same steps for the ‘Question’ rectangle.

Step 9:

Now it is time to add our images. These images will contain the Would You Rather questions. For this tutorial, I have collected 10 fun questions and made them into an image. Download them here.

You can have as many images/questions as you want for this filter. Remember to have the same file name for the images with a different number at the end eg: ques1, ques2, ques3, etc. We need to do this because we will be importing these images as an animation sequence.

To do this, click on the ‘add asset’ button at the bottom of the left assets panel and select ‘animation sequence.’

Then select the newly created animation sequence in the Assets panel and click on the choose file button under the texture property on the right panel.

Now, open the folder where all your images are stored and select them all together and click open. Once done, you should be able to see a new texture created in the assets panel named ques[1–10] or whatever you have named your image ranging from 1 to the number of images.

Note: To avoid loss of quality you can click on the texture and select ‘no compression.’ Remember that there is a 4Mb limit on the overall filter file and not using compression may lead to increased memory size for large image files.

Once you have the questions animation sequence imported, you can import the title screen image. Download it here and simply drag and drop it into the assets panel to import it.

Step 10:

At this point, we have imported all assets, and it’s time to make the questions appear. First, select the material you used on the ‘Title’ rectangle and set its texture to the title image you just imported. It may be possible that the image is not visible in the simulator if the ‘Title’ rectangle is located above the ‘Question’ rectangle in the scene hierarchy.

Next, set the texture property of the material you used on the ‘Question’ rectangle to the Animation Sequence we created earlier.

You will now see the questions appear on the rectangle but they are changing very quickly. By default, the animation is set to 25 frames per second(FPS) and to slow it down you can change the FPS property of the Animation Sequence to a lower number eg: 15. Feel free to adjust it to your liking. It is also a good idea to check the ‘Randomize’ option so that the animation frame images that contain our questions get shuffled.

Step 11:

It is finally time to level up and use the patch editor to add some logic to our effect. At this point, we have the banner visible over the user’s head and it has questions shuffling randomly but indefinitely.

The functionality we need to implement in this filter is to add a start and termination point. We can make the questions start shuffling as soon as the user starts recording and then stop at a random one after a few seconds. This is what most filters of this type do, however, you can always tweak it to your liking once we know how to do this.

The Patch Editor

The patch editor is what makes Spark AR different from its competitor Lens Studio for Snapchat. This editor consists of a drag and drop programming interface similar to that of Scratch (a tool for children to learn programming). This eliminates the requirement of prior programming experience which makes it the go-to tool for a lot of non-programmers to get into Augmented Reality and explore their creative side.

To activate the patch editor, go to the top menu and select View > Show/Hide Patch Editor.

This will open the Editor at the bottom of the screen.

The patch editor interface uses visual elements called ‘Patches’ that represent blocks of code. These patches can be linked with each other via connections to bring about functionality.

To make it super easy for beginners, Spark provides an in-app search & documentation for patches. Right-clicking inside the patch editor will open up a Search window for all the available patches.

At times when you need to get an object in the scene or an asset into the patch editor, you can do this by simply dragging them from the left into the patch editor and it will automatically be created as a patch.

All properties of an object that you were able to edit in the inspector panel visually can also be changed via the patch editor. To get a patch for a property, select the object and click on the arrow icon next to the property you want access to in the patch editor and Spark will create a patch for you.

Step 12:

To know which patches to use, let us go through the logic flow we need:

The ‘Title’ rectangle (we added in step 10) displays the initial question to the user by default when the effect is opened. When the user starts recording, the ‘Title’ rectangle is hidden and the ‘Question’ appears and the shuffling starts. After a few seconds, the shuffling stops and the user answers the question displayed on the screen.

Now, to start we need to know when the camera recording begins. To know this, we need to use the ‘Camera’ object’s patch in the patch editor. To get it in, simply drag the ‘Camera’ object from the Scene panel on the left into the patch editor. Alternately, you can select the ‘Camera’ object and click on the producer patch option in the Inspector pane.

Next, we need to change the visibility property of the ‘Questions’ rectangle. Now, in the logic flow earlier we mentioned that we want to hide the ‘Title’ rectangle, however, if you check our object hierarchy in the scene panel, the ‘Title’ rectangle comes above the ‘Questions’ rectangle, covering it by default.

So, if want to display the Title, we can hide the ‘Questions’ rectangle or the opposite, in our case.