Screen Space in ARKit (+tutorial)
Tags: arkit, swift, ios, iphone
Published: 2022-04-03 13:37:45

There is a lot of stuff in AR I would like to talk about, but let's not waste your time (or mine) and get right to it!

</div>![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eab1gzqh5zbg871y5rej.jpg)

This year at **WWDC**(2018), Grant Paul from the Human Interface team at Apple was talking about creating great AR experiences.
One thing that he really stressed out is the usage of **Screen Space** instead of **Perspective** for rendering text in our AR apps.

If you missed the lecture, I highly recommend giving it a whirl. It stresses out some key points that are really important, and ARKit being a new kid on the block, needs us as developers to get him through the hazing process.
<a href="https://developer.apple.com/videos/play/wwdc2018/805">Check it out here.</a>
</p>

---

## What is ScreenSpace?

Short answer --> Barrels in the original DOOM. I am not even kidding.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4z5lztc3m0neesmkeq5m.jpg) 

Basically, it's placing a 2D object in a 3D space. It really does have a 3D placement, but the object itself is a 2D sprite. We have loads of reasons to do this in iOS, but the main one is of course **readability.**

Take a look at this example used in the lecture. We see that the text that is placed with perspective, or *SCNText*, doesn’t work that well at all.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1fc655ux25mya71c5h7y.jpg) 

On the other hand, text placed in Screen Space is always the same size and facing towards the camera, making it perfect for placement of informational data.

I know I know, barrels in **Doom** do change the size at least, but still, way too good of a reference to be missing out on.

---

## So, HOW do we get there?
Below is a quick tutorial on how to utilise screen space in your projects. This is just a basic template project with no real usage but you should be able to get the basics of screen space and how it works.

### 1. Project Creation and Setup
Start a new project and select an AR App template.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gh4ju42yarvmj1fxmk6q.jpg) 

Whenever I start working with AR template, one thing that bothers me is not being able to add labels and views via storyboards.

### So, to solve that:
+ Remove the default ViewController
+ Add a new ViewController from the Object library
+ Add a UIView inside the new VC
+ And finally add an ARSCNView to the VC

Now you have a *UIView* that can hold all labels, texts, images, and anything your heart desires, but under all of that, you have a *ARSCNView* for your AR shenanigans.
If you have a better solution for this setup I would love to hear it. So far this works great for me and it’s easy-peasy.


Note: The setup above is not a <strong>MUST</strong> for this tutorial, just thought to share it with you for future AR-related stuff.
Don’t forget to remove the old outlet and create a new one for the <em>ARSCNView</em> you just placed!

---

### 2. Create a UIView and a Label
Next, in a separate swift file, define a class that conforms to the *UIView* and define a label inside that *UIView*.
Once we initialize the view later on, it will take a *SCNNode* as a required parameter so we need to define that as well.


```swift
import UIKit
import SceneKit

class SSLabelView: UIView {
  
  weak var labelNode: SSLabelNode!
  var label = UILabel()
  
  // Initializing view with a node
  init(labelNode: SSLabelNode) {
    
    self.labelNode = labelNode
    // creating a frame/size for the view
    super.init(frame: CGRect(origin: CGPoint.zero, size: CGSize(width: 100, height: 100)))
    
    // Adding a label to the view
    self.addSubview(label)
    
    // Adding constraints to the label
    label.translatesAutoresizingMaskIntoConstraints = false
    label.topAnchor.constraint(equalTo: self.topAnchor, constant: 10).isActive = true
    label.leadingAnchor.constraint(equalTo: self.leadingAnchor, constant: 10).isActive = true
    label.trailingAnchor.constraint(equalTo: self.trailingAnchor, constant: 10).isActive = true
    label.bottomAnchor.constraint(equalTo: self.bottomAnchor, constant: 10).isActive = true
    label.textColor = UIColor.white
    
    // Hardcodded text value
    label.text = "TEST"
  }
  
  required init?(coder aDecoder: NSCoder) {
    fatalError("Init Coder has not been initialized")
  }
}

class SSLabelNode: SCNNode {
}
```
---
### 3. Main VC need some love too!
In the main **ViewController** (i hope you connected it with the storyboard **you** messed up at the beginning 😅….sry, I know that one is on me 🤦‍♂️), declare an **array of views.**
Here, we will append all the views we create when we tap the screen. I know, sounds like **magic**, only less lame.

```swift
class ARViewController: UIViewController, ARSCNViewDelegate {

@IBOutlet weak var sceneView: ARSCNView!

var ssViews: [SSLabelView] = []
.
.
.
```

*viewDidLoad* needs to be updated from the template since we are not using that sweet plane provided by Apple, so here is the code for that.

```swift
override func viewDidLoad() {
    super.viewDidLoad()
    
    // Set the view's delegate
    sceneView.delegate = self
    
    // Show statistics such as fps and timing information
    sceneView.showsStatistics = true
    
    // Set the scene to the view
    sceneView.scene = SCNScene()
  
  // Set the Session delegate
  sceneView.session.delegate = self
}
```

Note, that at the very bottom I added a <strong>session delegate.</strong> This is crucial to make the app work, but if you write it NOW, you will get errors for sure. So if you decide to do so, just ignore the errors for bit and all shall be revealed!


---

### 4. Touchy touchy
Next, we need to see how touches are handled. We just use a simple *hitTest* to get the latest touch location, and forward that info to a function that we will create in the next step.

```swift
override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
  if let touch = touches.first {
    
    let touchLocation = touch.location(in: sceneView)
    let results = sceneView.hitTest(touchLocation, types: .featurePoint)
    
    if let hitResult = results.first {
      addLabel(at: hitResult)
    }
  }
}
```

---

### 5. Adding that label like a baws
Once we have the information of where the user tapped, we give that location to the *SCNNode* we defined at the start, add it as a subview and **append** it to the **array** we defined that will hold all the views that we place.

```swift
func addLabel(at tapLocation: ARHitTestResult) {
  
  let labelPosition = SCNVector3(tapLocation.worldTransform.columns.3.x,
                                 tapLocation.worldTransform.columns.3.y,
                                 tapLocation.worldTransform.columns.3.z)
  
  let ssLabelPosition = SSLabelNode()
  ssLabelPosition.worldPosition = labelPosition
  self.sceneView.scene.rootNode.addChildNode(ssLabelPosition)
  
  let ssLabelPlacedView = SSLabelView(labelNode: ssLabelPosition)
  self.view.addSubview(ssLabelPlacedView)
  self.ssViews.append(ssLabelPlacedView)
}
```

<p class="info">
I tried to keep the naming as clear as possible so it makes sense, but it is much easier once you rewrite it and use naming convention that works for you 👍
My naming convention is equal to the thought a drunk gives when ordering a new drink. <strong>Whatever works.</strong>
</p>

---

### 6. Almost there!
Now we are ready to finally place the views in our *ARSCNView*. We have defined nodes all over the place and everything is there, but not really there.

So, in a new swift file we create an extension to the *ViewController* that conforms to the *ARSessionDelegate*. We do that, so our view would be updated every frame -> 60 frames per second. Smoooooth.

We go inside every view from our **array** defined in the main VC, and give it a position smack middle of the feature point.

There is also a small fixer function, thanks to [Greg Chiste on YouTube][gregYTLink] , who has a video tutorial for screen space that helped me a **lot** getting the gist of it. That function makes sure that placed lables don’t appear behind us as well (prison bad UX right there if you know what I mean).

```swift
import ARKit

extension ARViewController: ARSessionDelegate {
  
  func session(_ session: ARSession, didUpdate frame: ARFrame) {
    
    for view in ssViews {
      let projectedPosition = self.sceneView.projectPoint(view.labelNode.worldPosition)
      onlyShowViewsInFront(projectedPosition: projectedPosition, view: view)
      let size = view.frame.size
      let x = CGFloat(projectedPosition.x) - size.width/2
      let y = CGFloat(projectedPosition.y) - size.height/2
      
      view.frame.origin = CGPoint(x: x, y: y)
    }
  }
  
  fileprivate func onlyShowViewsInFront(projectedPosition: SCNVector3, view: UIView) {
    if projectedPosition.z > 1 {
      view.isHidden = true
    } else {
      view.isHidden = false
    }
  }
}
```

---

## You in AR now son.
If you run the app, move the phone a bit to pick up the feature points, and then **smack** that screen!
Text “TEST” will be placed in your 3D space and when you move around, you should see that it stays the same size no matter the distance. Pretty neat ha? 😀

---

Thank you for reading! Feel free to get in touch on [Twitter][twitter]

Consider subscribing to the newsletter below if you loved the content.
I am so lazy I probably won't bore you more than once a week.

Cheers! 🍻

[gregYTLink]: https://www.youtube.com/channel/UC2hlc4dPZNBPuAfZ1iqgmZg
[twitter]: https://twitter.com/emin_ui