Implementing Flutter QR Code Scanner with Swift and AVFoundation for iOS
Tags: flutter, ios, swift, dart
Published: 2024-05-08 05:56:10

In the previous article, we implemented a Flutter barcode and QR code scanner for Android using **Kotlin** and **CameraX**. Since the **Dart** code is platform-independent, no changes are necessary. In this article, we will take steps to implement the native camera and barcode scanning logic for iOS using **Swift**, **AVFoundation**, and the **Dynamsoft Barcode Reader SDK**.

## Prerequisites
- [Dynamsoft Barcode Reader Trial license](https://www.dynamsoft.com/customer/license/trialLicense/?product=dbr) 


## Step 1: Installing Dynamsoft Barcode Reader for iOS
We use **CocoaPods** to install the Dynamsoft Barcode Reader SDK for iOS. If you haven't installed CocoaPods yet, please follow the official instructions [here](https://guides.cocoapods.org/using/getting-started.html) to do so.


Once CocoaPods is ready, create a `Podfile` in the iOS folder of your Flutter project:

```bash
cd ios
pod init
```

Next, edit the `Podfile` to include the Dynamsoft Barcode Reader SDK:

```ruby
target 'Runner' do
  use_frameworks!

  pod 'DynamsoftBarcodeReader','9.6.40'

end
```

Save the `Podfile` and run `pod install`. This command will install or update the CocoaPods dependencies, including the Flutter framework required for your iOS project.

## Step 2: Adding Camera Permission to Info.plist
To enable camera access on iOS, open the `Info.plist` file located in the `ios/Runner` folder. Add the following keys:

```xml
<key>NSCameraUsageDescription</key>
<string>your usage description here</string>
<key>NSMicrophoneUsageDescription</key>
<string>your usage description here</string>
```

## Step 3: Implementing Camera Preview with Flutter Texture in Swift
The `Runner/AppDelegate.swift` file serves as the entry point of the Flutter application. By default, it contains the following boilerplate code:

```swift
override func application(
    _ application: UIApplication,
    didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
  ) -> Bool {
    GeneratedPluginRegistrant.register(with: self)
    return super.application(application, didFinishLaunchingWithOptions: launchOptions)
  }
```

To integrate the camera functionality effectively, you'll need to enhance the application method with the following steps:
1. **Establish a Flutter method channel**. This is crucial for seamless communication between the Dart environment and Swift, allowing commands and data to be exchanged between the Flutter UI and native code.
2. **Implement a startCamera() method**. This method should initiate the camera preview and continuously render this preview into a Flutter texture. This involves setting up the camera capture session, configuring input and output, and linking the camera output to a Flutter texture that can be displayed in the UI.

### Flutter Method Channel in Swift
The Flutter method channel is a named channel that facilitates the sending of data between Dart and platform-specific code.

```swift
private var channel: FlutterMethodChannel?
private var width = 1920
private var height = 1080

override func application(
    _ application: UIApplication,
    didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
  ) -> Bool {

    channel = FlutterMethodChannel(name: CHANNEL, binaryMessenger: flutterViewController.binaryMessenger)
    channel?.setMethodCallHandler({
      (call: FlutterMethodCall, result: @escaping FlutterResult) -> Void in
      if call.method == "startCamera" {
        self.startCamera(result: result)
      } else if call.method == "getPreviewWidth" {
        result(self.width)
      } else if call.method == "getPreviewHeight" {
        result(self.height)
      }
      
      else {
        result(FlutterMethodNotImplemented)
      }
    })

    GeneratedPluginRegistrant.register(with: self)
    return super.application(application, didFinishLaunchingWithOptions: launchOptions)
  }
```
- The `channel` variable is an instance of `FlutterMethodChannel`. It receives method calls from `Dart` using the `setMethodCallHandler` method. Additionally, the `invokeMethod` method is used to send data from `Swift` to `Dart`.
- The `width` and `height` variables, which store the camera preview size, are hardcoded to `1920x1080` here. The methods `getPreviewWidth` and `getPreviewHeight` retrieve and return these values to Dart, respectively.


### Creating Flutter Texture and Camera Preview
Define the `CustomCameraTexture` class that extends `NSObject` and implements the `FlutterTexture` protocol:

```swift
class CustomCameraTexture: NSObject, FlutterTexture {
  private weak var textureRegistry: FlutterTextureRegistry?
  var textureId: Int64?
  private var cameraPreviewLayer: AVCaptureVideoPreviewLayer?
  private let bufferQueue = DispatchQueue(label: "com.example.flutter/barcode_scan")
  private var _lastSampleBuffer: CMSampleBuffer?
  private var customCameraTexture: CustomCameraTexture?

  private var lastSampleBuffer: CMSampleBuffer? {
    get {
      var result: CMSampleBuffer?
      bufferQueue.sync {
        result = _lastSampleBuffer
      }
      return result
    }
    set {
      bufferQueue.sync {
        _lastSampleBuffer = newValue
      }
    }
  }

  init(cameraPreviewLayer: AVCaptureVideoPreviewLayer, registry: FlutterTextureRegistry) {
    self.cameraPreviewLayer = cameraPreviewLayer
    self.textureRegistry = registry
    super.init()
    self.textureId = registry.register(self)
  }

  func copyPixelBuffer() -> Unmanaged<CVPixelBuffer>? {
    guard let sampleBuffer = lastSampleBuffer, let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else {
      return nil
    }

    return Unmanaged.passRetained(pixelBuffer)
  }

  func update(sampleBuffer: CMSampleBuffer) {
    lastSampleBuffer = sampleBuffer
    textureRegistry?.textureFrameAvailable(textureId!)
  }

  deinit {
    if let textureId = textureId {
      textureRegistry?.unregisterTexture(textureId)
    }
  }
}
```

- The `textureRegistry` variable is an instance of `FlutterTextureRegistry`. It is used to register and unregister the Flutter texture.
- The `textureId` variable stores the Flutter texture ID, which will be used to render the camera preview in Flutter.
- The `copyPixelBuffer()` method returns the latest pixel buffer for texture rendering.
- The `update()` method appends a new camera frame and then notifies the Flutter texture that it needs to be updated. When the `textureFrameAvailable()` method is invoked, the `copyPixelBuffer()` method is triggered to fetch the latest pixel buffer.

Create a `flutterTextureEntry` variable in the `AppDelegate` class and obtain the Flutter texture registry within the `application` method:

```swift
private var flutterTextureEntry: FlutterTextureRegistry?

override func application(
    _ application: UIApplication,
    didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
  ) -> Bool {
    ...

    guard let flutterViewController = window?.rootViewController as? FlutterViewController else {
      return super.application(application, didFinishLaunchingWithOptions: launchOptions)
    }
    flutterTextureEntry = flutterViewController.engine!.textureRegistry
    
    GeneratedPluginRegistrant.register(with: self)
    return super.application(application, didFinishLaunchingWithOptions: launchOptions)
  }

```


In the `startCamera()` initiate a camera session and add a video data output to capture the camera frames. When a new frame is captured in the `captureOutput()` method, update the `CustomCameraTexture` instance with the latest sample buffer:

```swift
private func startCamera(result: @escaping FlutterResult) {
    if cameraSession != nil {
      result(self.customCameraTexture?.textureId)
      return
    }

    cameraSession = AVCaptureSession()
    cameraSession?.sessionPreset = .hd1920x1080

    guard let backCamera = AVCaptureDevice.default(for: .video), let input = try? AVCaptureDeviceInput(device: backCamera) else {
      result(FlutterError(code: "no_camera", message: "No camera available", details: nil))
      return
    }

    cameraSession?.addInput(input)
    cameraPreviewLayer = AVCaptureVideoPreviewLayer(session: cameraSession!)
    cameraPreviewLayer?.videoGravity = .resizeAspectFill

    let cameraOutput = AVCaptureVideoDataOutput()
    cameraOutput.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA]
    cameraOutput.setSampleBufferDelegate(self, queue: DispatchQueue(label: "camera_frame_queue"))
    cameraSession?.addOutput(cameraOutput)

    self.customCameraTexture = CustomCameraTexture(cameraPreviewLayer: cameraPreviewLayer!, registry: flutterTextureEntry!)
    cameraSession?.startRunning()

    result(self.customCameraTexture?.textureId)
  }

  func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
    if connection.isVideoOrientationSupported {
      connection.videoOrientation = currentVideoOrientation()
    }
    self.customCameraTexture?.update(sampleBuffer: sampleBuffer)
  }
```

At this point, the camera preview should function correctly on iOS/iPadOS. Next, we will integrate the Dynamsoft Barcode Reader SDK to decode barcodes and QR codes.

## Step 4: Integrating Dynamsoft Barcode Reader SDK for iOS in Swfit
1. Import the Dynamsoft Barcode Reader SDK in the `AppDelegate.swift` file:

    ```swift
    import DynamsoftBarcodeReader
    ```

2. Create an instance of Dynamsoft Barcode Reader and activate it with a valid license key:

    ```swift
    @UIApplicationMain
    @objc class AppDelegate: FlutterAppDelegate, AVCaptureVideoDataOutputSampleBufferDelegate, DBRLicenseVerificationListener {

    private let reader = DynamsoftBarcodeReader()

    override func application(
        _ application: UIApplication,
        didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
      ) -> Bool {
        DynamsoftBarcodeReader.initLicense("LICENSE-KEY", verificationDelegate: self)
        
        do {
          let settings = try? reader.getRuntimeSettings()
          settings!.expectedBarcodesCount = 999
          try reader.updateRuntimeSettings(settings!)
        } catch {
          print("Error getting runtime settings")
        }
        
        ...

        GeneratedPluginRegistrant.register(with: self)
        return super.application(application, didFinishLaunchingWithOptions: launchOptions)
      }
    
        func dbrLicenseVerificationCallback(_ isSuccess: Bool, error: Error?) {
            if isSuccess {
            print("License verification passed")
            } else {
            print("License verification failed: \(error?.localizedDescription ?? "Unknown error")")
            }
        }
    }
    ```

3. Decode barcode and QR code from the camera frame in the `captureOutput()` method. Since the decoding API is CPU-intensive, to avoid blocking the camera preview rendering, we move the decoding logic to a separate thread:

    ```swift
    func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
        if connection.isVideoOrientationSupported {
          connection.videoOrientation = currentVideoOrientation()
        }
        self.customCameraTexture?.update(sampleBuffer: sampleBuffer)
    
        if !isProcessing {
          isProcessing = true
          DispatchQueue.global(qos: .background).async {
            self.processImage(sampleBuffer)
            self.isProcessing = false
          }
        }
      }
    ```
    
    The `isProcessing` boolean variable ensures that only one frame is processed at a time, and new frames are ignored until the processing is complete. This approach helps mitigate the accumulation of asynchronous tasks and prevents the app from crashing due to memory exhaustion.


4. Implement the `processImage()` method to decode barcodes from `CMSampleBuffer` and send the results to the Flutter UI via the method channel:

    ```swift
    func processImage(_ sampleBuffer: CMSampleBuffer) {
        let imageBuffer:CVImageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!
        CVPixelBufferLockBaseAddress(imageBuffer, .readOnly)
        let baseAddress = CVPixelBufferGetBaseAddress(imageBuffer)
        let bufferSize = CVPixelBufferGetDataSize(imageBuffer)
        let width = CVPixelBufferGetWidth(imageBuffer)
        let height = CVPixelBufferGetHeight(imageBuffer)
        let bpr = CVPixelBufferGetBytesPerRow(imageBuffer)
        CVPixelBufferUnlockBaseAddress(imageBuffer, .readOnly)
        let buffer = Data(bytes: baseAddress!, count: bufferSize)
        
        let imageData = iImageData.init()
        imageData.bytes = buffer
        imageData.width = width
        imageData.height = height
        imageData.stride = bpr
        imageData.format = .ARGB_8888
        imageData.orientation = 0
    
        let results = try? reader.decodeBuffer(imageData)
        DispatchQueue.main.async {
          self.channel?.invokeMethod("onBarcodeDetected", arguments: self.wrapResults(results: results))
        }
      }
    
      func wrapResults(results:[iTextResult]?) -> NSArray {
            let outResults = NSMutableArray(capacity: 8)
            if results == nil {
                return outResults
            }
            for item in results! {
                let subDic = NSMutableDictionary(capacity: 11)
                if item.barcodeFormat_2 != EnumBarcodeFormat2.Null {
                    subDic.setObject(item.barcodeFormatString_2 ?? "", forKey: "format" as NSCopying)
                }else{
                    subDic.setObject(item.barcodeFormatString ?? "", forKey: "format" as NSCopying)
                }
                let points = item.localizationResult?.resultPoints as! [CGPoint]
                subDic.setObject(Int(points[0].x), forKey: "x1" as NSCopying)
                subDic.setObject(Int(points[0].y), forKey: "y1" as NSCopying)
                subDic.setObject(Int(points[1].x), forKey: "x2" as NSCopying)
                subDic.setObject(Int(points[1].y), forKey: "y2" as NSCopying)
                subDic.setObject(Int(points[2].x), forKey: "x3" as NSCopying)
                subDic.setObject(Int(points[2].y), forKey: "y3" as NSCopying)
                subDic.setObject(Int(points[3].x), forKey: "x4" as NSCopying)
                subDic.setObject(Int(points[3].y), forKey: "y4" as NSCopying)
                subDic.setObject(item.localizationResult?.angle ?? 0, forKey: "angle" as NSCopying)
                subDic.setObject(item.barcodeBytes ?? "", forKey: "barcodeBytes" as NSCopying)
                outResults.add(subDic)
            }
    
            return outResults
        }
    ```

## Running the Flutter QR Code Scanner on iOS

```bash
flutter run
```

![Flutter iOS QR code scanner](https://www.dynamsoft.com/codepool/img/2024/05/ios-camera-flutter-qr-code-scanner.png)

## Source Code
[https://github.com/yushulx/flutter-barcode-scanner/tree/main/examples/native_camera](https://github.com/yushulx/flutter-barcode-scanner/tree/main/examples/native_camera)