Managing iOS Augmented Reality Views with Objective-C in React Native Apps
Tags: reactnative, ios, objectivec, arkit
Published: 2023-12-31 13:02:53

## Introduction

Integrating 3D models into AR applications can significantly enhance the user experience by providing interactive and immersive elements. For iOS applications, using _Apple's ARKit_ with the .usdz file format is often the preferred approach due to its native support and optimization for AR experiences. This introduction will guide you through the steps of integrating a 3D model into an iOS application using _ARKit_.

This is part four of a multi part article series about the integration of AR services via native modules into a _React Native_ app. While React Native simplifies cross-platform app development, there are scenarios where leveraging native iOS capabilities becomes essential, especially when dealing with advanced features like augmented reality.

You can find the other parts here:

- Part one: [Setting up a native module via Kotlin](https://dev.to/9bytes/building-ar-vr-experiences-with-react-native-part-1-bridging-the-gap-with-kotlin-1dii).
- Part Two: [Integrating ARCore](https://dev.to/9bytes/navigating-the-ar-landscape-in-react-native-with-kotlin-2caj)
- Part Three: [Setting up a native module via Objective-C](https://dev.to/9bytes/seamlessly-integrating-native-modules-in-ios-for-react-native-ar-apps-5alp)

The full code for this part is [here](https://github.com/Gh05d/RN3DWorldExplorer/tree/part-4).

## 3D-Model integration

Unfortunately, using _GLB_ files in _iOS_ AR applications is a bit of a hustle, so it is easier for us the use another format for the _iOS_ version, namely _USDZ_, instead of using the already obtained _GLB_ file from the _Android_ version in [part 2](https://dev.to/9bytes/navigating-the-ar-landscape-in-react-native-with-kotlin-2caj) of this series.

### Download a model

Basically, download any model you like from a site like [this](https://developer.apple.com/augmented-reality/quick-look/). I downloaded the pancakes model.

### XCode

The next step is to add the downloaded model to XCode. Naturally, with all things _Apple_, this is far more annoying than it has any right to be:

1. Right-click on the project's root, select _New Group_ and name it _Resources_.
2. Drag the .usdz file from Finder and drop it into the _Resources_ group in Xcode's project navigator.
3. When you add the file, make sure the options are set correctly in the dialog that appears. Ensure the _Copy items if needed_ is checked. Also, ensure that the file is added to the app's target by checking _RN3DWorldExplorer_, so it's included in the build.

### Update info.plist

The last step is to add the following key to your info.plist file (located at _ios/RN3dWorldExplorer_):

```xml
<key>NSCameraUsageDescription</key>
<string>This app needs access to the camera to create 3D AR experiences.</string>
```

This is needed as we access the users camera for the 3D-Model, which _Apple_ regards as privacy-sensitive data.

## Coding

We will be doing several updates to our _RCTARodule.m_ file in order to use _ARKit_. At first, add the following imports to the top of the file to leverage the AR capabilities on _iOS_:

```objective-c
#import <ModelIO/ModelIO.h>
#import <SceneKit/ModelIO.h>
#import <ARKit/ARKit.h>
```

Now we will be implementing a new interface. Copy this code between the imports and the `@implementation RCTARModule`:

```objective-c
@interface RCTARModule () <ARSCNViewDelegate>
@property (nonatomic, strong) ARSCNView *sceneView;
@property (nonatomic, strong) SCNNode *modelNode;
@end
```

In essence, this code is preparing the `RCTARModule` class to handle AR functionalities. It's setting up an AR SceneView (`ARSCNView`) to display AR content and a node (`SCNNode`) to load our 3D model in the AR scene. The module is also prepared to handle `ARSCNView` events by conforming to the `ARSCNViewDelegate` protocol.

We will update our `showAR` function in the now to be able to display the model. This will take several steps. Exchange the current code for this:

```objective-c
RCT_EXPORT_METHOD(showAR:(NSString *)filename)
{
  dispatch_async(dispatch_get_main_queue(), ^{
    RCTLogInfo(@"Loading model: %@", filename);
    [self initializeARView];
    [self loadAndDisplayModel:filename];
    [self presentARView];
  });
}
```

Now we are logging the passed in parameter `filename` before we are calling three functions to set up the AR view, load and display the model and present the view to the user in the end. Don't get shocked by the amount of errors popping up, they will go away as we are implementing the functions.

### initializeARView

Use this code to implement the function:

```objective-c
- (void)initializeARView {
  self.sceneView = [[ARSCNView alloc] initWithFrame:UIScreen.mainScreen.bounds];
  self.sceneView.delegate = self;

  // Configure AR session
  ARWorldTrackingConfiguration *configuration = [ARWorldTrackingConfiguration new];
  configuration.planeDetection = ARPlaneDetectionHorizontal;
  [self.sceneView.session runWithConfiguration:configuration];
}
```

In general, the `initializeARView` method sets up the AR environment for the application. It initializes an `ARSCNView` to render the AR content, sets up the class as its delegate to handle AR-related events, configures the AR session to include horizontal plane detection, and finally starts the AR session with these configurations.

### loadAndDisplayModel

Now implement the function with this code:

```objective-c
- (void)loadAndDisplayModel:(NSString *)filename {
  NSString *filePath = [[NSBundle mainBundle] pathForResource:filename ofType:@"usdz"];
  NSURL *fileURL = [NSURL fileURLWithPath:filePath];

  NSError *error = nil;
  SCNScene *scene = [SCNScene sceneWithURL:fileURL options:nil error:&error];

  // Correctly set the modelNode property
  self.modelNode = [scene.rootNode.childNodes firstObject];
  if (self.modelNode) {
    self.modelNode.scale = SCNVector3Make(0.1, 0.1, 0.1);
    self.modelNode.position = SCNVector3Make(0, -1, -3); // Adjust as needed
    [self.sceneView.scene.rootNode addChildNode:self.modelNode];
  }
}
```

This method is designed to load a 3D model from the application's main bundle and display it in the AR scene. First it constructs a file path for a 3D model file (usdz format) based on the provided filename. Then, it creates a URL (fileURL) pointing to this file. After that, it creates a `SCNScene` and adds the model node to the AR scene. Be aware that when you use your own model, you probably want to adjust the `scale` and `position`, as it could be too near / far from the camera.

### presentARView

Finally, add these two functions to the file:

```objective-c
- (void)presentARView {
    UIViewController *viewController = [UIViewController new];
    [viewController.view addSubview:self.sceneView];

  // Create a close button
  UIButton *closeButton = [UIButton buttonWithType:UIButtonTypeSystem];
  [closeButton setTitle:@"X" forState:UIControlStateNormal];
  [closeButton addTarget:self action:@selector(closeARView) forControlEvents:UIControlEventTouchUpInside];

  CGFloat buttonSize = 44.0;
  CGFloat padding = 16.0;
  closeButton.frame = CGRectMake(viewController.view.bounds.size.width - buttonSize - padding,
     padding,
     buttonSize,
     buttonSize);
  closeButton.autoresizingMask = UIViewAutoresizingFlexibleLeftMargin | UIViewAutoresizingFlexibleBottomMargin;
  closeButton.backgroundColor = [UIColor blueColor];
  closeButton.layer.cornerRadius = buttonSize / 2;
  closeButton.clipsToBounds = YES;

  [viewController.view addSubview:closeButton];

    // Assuming you have access to the root view controller or the current view controller
    UIViewController *rootViewController = RCTPresentedViewController();
    [rootViewController presentViewController:viewController animated:YES completion:nil];
}

- (void)closeARView {
    UIViewController *presentingController = self.sceneView.window.rootViewController;
    [presentingController dismissViewControllerAnimated:YES completion:nil];
}
```

The first half of the `presentARView` function just adds a close button to the view and assigns the helper function `closeARView` to it. The latter part of the method concludes by presenting the new view controller modally on top of the current view hierarchy. This is done by retrieving the current application's presented view controller (`RCTPresentedViewController()`) and calling `presentViewController:animated:completion:` on it.

### Update App.tsx

Now we finally move to our _React Native_ code. Update the `Button` s `onPress` method:

```jsx
await ARModule.showAR(Platform.OS == "ios" ? "pancakes" : "AR-model.glb");
```

If you use a different model, exchange pancakes for the file name. Don't forget to `import { Platform } from "react-native"` at the top of the file. Start up the application and test the code.

## Conclusion

Successfully integrating a 3D model into an iOS AR application involves several critical steps, from preparing the appropriate file format to coding the interaction with _ARKit_. While the process might seem daunting initially, especially due to platform-specific requirements like using .usdz files for _iOS_, the result is a compelling and immersive AR experience.

This concludes this series for now. Eventually there will be some articles about the integration of _VR_ functionality like 360 videos for both platforms, but the setup process is way more complicated than anything explored in this series till now. For example on Android, it involves downloading the _Cardboard SDK_ and surgically integrating it into the Android portion of the app via generating .proto files and .java files.

By now, you should have a basic understanding of the integration of _AR_ services into _React Native_ apps. Your next steps would the integration of advanced functionality like moving the models around or scaling them by using pinch gestures. The sky is the limit from here now on.