Zero to Hero With Spark AR
Tags: Augmented Reality, Beginners Guide, AR, Beginner, Spark Ar

If you’re new to Spark AR, this is for you: You will learn to go from newbie to pro!

Download Spark AR (Mac & Windows users) https://sparkar.facebook.com/ar-studio/download/ and follow installation instructions Launch Spark Ar You should see the welcome screen with templates but for this tutorial, click on New Project → Blank project

Welcome Screen

3. Have a look at the interface

Interface

4. Let’s start creating!!! We will start with the eyes and face. Creating Textures

Face

Download face assets: https://sparkar.facebook.com/ar-studio/learn/articles/people-tracking/face-reference-assets#whats-included-in-the-face-reference-assets These include meshes & textures to help you create masks. I use Photoshop but you can use your preferred editor.

Import the faceFeminine.png and the faceMeshTrackers.png (Notice how the tracker aligns) Create a new layer and make sure it is at the top & is selected. Choose a soft brush and white color and click on the left & right cheeks till you get your preference Disable visibility on all layers but layer 2 & save as a png file

Face in photoshop

Eyes

I used this https://unsplash.com/photos/B0zAPSrEcFw?utm_source=unsplash&utm_medium=referral&utm_content=creditShareLink

Open it in Photoshop Use the Elliptical Marque tool to cut out the iris by holding shift as you create the circle & place it on the iris. Right-Click, select inverse & delete Use the eraser set to soft round and erase the pupil. If you like this eye color, you can save it as a png, if not: Click on Image → Adjustments → Desaturate On the properties tab, adjust the RGB level, use the sharpener tool on the lighter part then save as a png

Back to Spark Ar

Click on “Add Object” in the scene panel and add a Face tracker Right-click on the face tracker, Add a null object & rename it to “Eyes” Right-click on the null-object, add two null objects & rename to “Left” & “Right” Right-click on “Left”, Add a plane. A panel appears on the right. On Scale, change x & y to 0.16. (Do the same for “Right”). Rename the planes to “leftEye” & “rightEye” On the top toolbar, click on View → show/hide patch editor (Patch editor window opens) Click & drag the face tracker to the patch editor. The patch editor is quite easy. Hover around any of the outputs and it displays what each should be connected to instead of writing code. Even if you connect an output to a wrong input, it will display an error and tell you exactly why. In our case, we want to track the face to the eyeball. On the face tracker output, drag the first output, “face” a little & release. A window appears. search for eyeball and click. Notice that the face tracker output for face connects to the face input for eyeball. (Easy right?) We want the position & rotation of the iris to automatically size with the eyeball. Click on “leftEye”(plane) and on the arrows on the right panel for the position & rotation. They appear in the patch editor. Connect left eye from the eyeball to left eye position & the left eye rotation eyeball output to the left eye rotation. (Repeat for “rightEye” plane but connecting to the right). The planes now move to the eyes. Click on the leftEye plane and the + materials on the right plane. Rename this material(Assets panel) to “Eye”. For the right eye plane, click on materials & on “Eye” as they’ll use the same one. Click on material, on the right pane, change shader type to “flat”, on render options, change Alpha to Add, Lower opacity to 60% & on Advanced Render options, uncheck the use & write to depth. On diffuse, click on texture & select, eye. Notice the pupils change. If you used the black & white texture, click on color & choose your preferred color. You can go back to the planes and resize the iris’ to your desire. Your project structure should look like this

You want the colors not to be visible when you close your eyes, so just add this, by linking the face output of the facetracker to the plane visibility. (click on the lefteye & the arrow next to visibility on the right pane, & the same for the righteye plan) Add the not & connect it to the visibility. Logically reading it, if the eye is not closed, the plane is visible. (Patch editor is easy, right?)

Patch Editor

Face Mask

Right-click on Face Tracker and Add Face mesh. Click on the +material, create a new material called, blush. Click on blush material, change shader type to flat, on Advanced render options, uncheck the use & write to depth test. On diffuse, choose the face texture we created earlier! (See the highlight?) To change this color, click on color & choose your preference. Reduce the opacity to 8% or to your liking.

Output

Cheers to making it this far!

previewing & uploading

Hair

On the workspace, add object → Add rectangle & rename it to hair. It comes as a child of the canvas the same way the face mesh is a child of the face tracker. Add material and rename it to hair. Change shader type to flat, on the render options, change alpha to add, reduce the opacity to 60% & on the advanced render options, uncheck the use & write depth. Click on camera on the workspace and on the right pane, click on the segmentation + and choose hair segmentation. Back to the hair material, check the alpha box on the right pane and choose hair segmentation as the texture. In the assets panel, click on import from the AR library and search for adjust colors shader. Import, then drag into the patch editor. We will need an input value for the texture and an output value for the final result. Right-click, search for value & insert. We want to change the hair color thus click on the value and on the dropdown, change number to color. Connect this to texture. Click on the hair material and select the arrow on diffuse to bring it into the patch editor. Then connect the output value of the adjust colors shader to it. Change the color value to your preference & notice the change in the preview window!

hair

What if you want the color to change on tap?

Repeat the value → adjustcolorsshader two more times. Have different colors for the value. Right-click and insert an option picker. Click on the wrench button → number of inputs and pick 3. (Notice the inputs are 4: the first is the index & will be explained in a bit) Connect each adjust colors shader to the option picker (there might be an error if all 3 are not connected as the picker is receiving less values than expected) And now to enable, “tap to change” Right-click and insert a picker UI. (Make sure visible is checked) The first output is the selected option index meaning that it should connect to the option picker index from above. To view the picker UI, we can add some colors (All I did was fill the page with the paint bucket). Import to Spark AR and drag into the patch editor & connect to the picker UI. It should be visible on your preview window

patch editor

Click on the three tiled menu on the preview window and select simulate on touch & tap on each to change color. Add more options if you wish!

final hair

3D Objects

On the assets panel, click on import from 3D library. Search for sunglasses & import. Import the headocculder.obj to Spark AR (Remember the assets we downloaded earlier?) The occulder is used to help you position 3D objects

position, rotate & scale

Drag both to your face tracker. For the head occluder, create new material and reduce this material's opacity so as to be able to resize the occluder to align to the head and now you can fit your sunglasses!

3D Objects

Recap

You’ve learned to :

Navigate through Spark AR Studio Create any own masks(2D, makeup, etc) from scratch Use the patch editor Use 3D objects If it ever feels too difficult, the templates are on the welcome screen.

Zero to Hero?