Implementing Flutter Barcode Scanner with Kotlin and CameraX for Android
Tags: flutter, android, kotlin, camera
Published: 2024-04-28 09:17:05

When considering the creation of a Flutter barcode scanner application, your first thought might be to search for an existing Flutter plugin. One option is to combine the **camera plugin** with a barcode scanning plugin, or to utilize a **barcode scanning plugin** that includes a camera preview feature. However, the camera plugin may **not deliver optimal performance** for image processing due to the memory copy process between native and Dart code. On the other hand, a barcode scanning plugin with a camera preview may **lack the flexibility** needed to access camera preview data for other purposes.

Therefore, to achieve **both optimal performance and flexibility** in camera-related applications, it may be preferable to **incorporate native code** for easy customization within a Flutter project. In this article, I will guide you through the step-by-step process of implementing a Flutter barcode scanner using **Kotlin**, **CameraX**, and a third-party image processing SDK—**Dynamsoft Barcode Reader**—for Android.


## Prerequisites
- [Dynamsoft Barcode Reader Trial license](https://www.dynamsoft.com/customer/license/trialLicense/?product=dbr) 


## Step 1: Scaffolding a Flutter Project
We use the **Flutter CLI** to create a new Flutter project:

```bash
flutter create <app_name>
```

To enable camera access, update the **AndroidManifest.xml** file with the required permissions and features:

```xml
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
    <uses-permission android:name="android.permission.CAMERA" />
    <uses-feature android:name="android.hardware.camera.any" />
    <uses-feature android:name="android.hardware.camera.autofocus" />
    
    <application>
        ...
    </application>
</manifest>
```

In this project, we will utilize `CameraX` for handling the camera preview and `Dynamsoft Barcode Reader` for decoding barcodes. To integrate these dependencies:

1. Set up the self-hosted Maven repository for Dynamsoft Barcode Reader in the **android/build.gradle** file:
    
    ```gradle
    allprojects {
        repositories {
            google()
            mavenCentral()
            maven {
                url "https://download2.dynamsoft.com/maven/aar"
            }
        }
    }
    ```

2. Include the dependencies in the **android/app/build.gradle** file:

    ```gradle
    dependencies {
        def camerax_version = "1.2.2"
        implementation "androidx.camera:camera-core:${camerax_version}"
        implementation "androidx.camera:camera-camera2:${camerax_version}"
        implementation "androidx.camera:camera-lifecycle:${camerax_version}"
        implementation "androidx.camera:camera-video:${camerax_version}"
    
        implementation "androidx.camera:camera-view:${camerax_version}"
        implementation "androidx.camera:camera-extensions:${camerax_version}"
    
        implementation 'com.dynamsoft:dynamsoftbarcodereader:9.6.40'
    }
    ```

    Note: The CameraX version may affect the compatibility with other dependencies. You can check the latest version on the [CameraX release page](https://developer.android.com/jetpack/androidx/releases/camera).

## Step2: Establishing Communication Between Dart and Kotlin
The **MethodChannel** class is used to facilitate interoperability between **Dart** and **Kotlin**. After instantiating the channel with a specific name in both **Dart** and **Kotlin**, messages can be sent and received between the two languages using `invokeMethod` and `setMethodCallHandler` methods.

**CameraPreviewScreen.dart**

```dart
class _CameraPreviewScreenState extends State<CameraPreviewScreen> {
  static const platform = MethodChannel('barcode_scan');

  @override
  void initState() {
    super.initState();
    platform.setMethodCallHandler(_handleMethod);
  }

  Future<dynamic> _handleMethod(MethodCall call) async {

  }
}
```

**MainActivity.kt**

```kotlin
class MainActivity : FlutterActivity(), ActivityAware {
    private val CHANNEL = "barcode_scan"
    private lateinit var channel: MethodChannel
    
    override fun configureFlutterEngine(flutterEngine: FlutterEngine) {
        channel = MethodChannel(flutterEngine.dartExecutor.binaryMessenger, CHANNEL)
        
        channel.setMethodCallHandler { call, result ->
   
        }
    }
}
```

## Step 3: Integrating CameraX into Flutter Android Project
To get started with CameraX in Android, you can refer to the following sample code:

- [https://github.com/flutter/packages/tree/main/packages/camera/camera_android_camerax](https://github.com/flutter/packages/tree/main/packages/camera/camera_android_camerax)
- [https://github.com/android/camera-samples/tree/main/CameraXBasic](https://github.com/android/camera-samples/tree/main/CameraXBasic)

In Flutter, there are two options for displaying a camera preview: using a `Texture` through a `TextureId` or embedding native views directly using `registerViewFactory` with a platform view.

**Using a TextureId**
1. The native code renders the camera preview (or other content) to a `SurfaceTexture` allocated in Android.
2. The ID of this `SurfaceTexture` is passed to Flutter, which uses it to create a `Texture` widget.
3. Flutter then displays the `Texture` widget, showing whatever is rendered on the associated `SurfaceTexture`.

**Using a Platform View**
1. Define a native view in Android, such as a `CameraView` or any custom view.
2. Register this view with Flutter's platform view system by creating a  `PlatformViewFactory` and linking it via `registerViewFactory`.
3. In Flutter, use a `PlatformView` (like `AndroidView` or `UiKitView` for iOS) to directly embed the native view within the Flutter widget tree.

For high-performance rendering, the `Texture` approach is recommended for displaying the camera preview in Flutter. Below is an outline of how to render the camera preview to a `Texture`.

**MainActivity.kt**
1. Define class variables for the camera preview and image analyzer:

    ```kotlin
    private val CAMERA_REQUEST_CODE = 101
    private val CHANNEL = "barcode_scan"
    private lateinit var channel: MethodChannel
    private lateinit var flutterTextureEntry: SurfaceTextureEntry
    private lateinit var flutterEngine: FlutterEngine
    private var lensFacing: Int = CameraSelector.LENS_FACING_BACK
    private var preview: Preview? = null
    private var imageAnalyzer: ImageAnalysis? = null
    private lateinit var cameraExecutor: ExecutorService
    private var camera: Camera? = null
    private var previewWidth = 1280
    private var previewHeight = 720
    ```

2. Request camera permission when configuring the Flutter engine:
    
    ```kotlin
    
    override fun configureFlutterEngine(flutterEngine: FlutterEngine) {
        ...

        requestCameraPermission()

        ...
    }

    private fun requestCameraPermission() {
        if (ContextCompat.checkSelfPermission(this, android.Manifest.permission.CAMERA) !=
                        PackageManager.PERMISSION_GRANTED
        ) {
            ActivityCompat.requestPermissions(
                    this,
                    arrayOf(android.Manifest.permission.CAMERA),
                    CAMERA_REQUEST_CODE
            )
        }
    }
    ```
   
3. Define an `ImageAnalyzer` class for processing images on a separate thread:

    ```kotlin
    private class ImageAnalyzer(listener: ResultListener? = null) : ImageAnalysis.Analyzer {
        private val listeners = ArrayList<ResultListener>().apply { listener?.let { add(it) } }

        private fun ByteBuffer.toByteArray(): ByteArray {
            rewind() 
            val data = ByteArray(remaining())
            get(data) 
            return data 
        }

        override fun analyze(image: ImageProxy) {
            
            if (listeners.isEmpty()) {
                image.close()
                return
            }

            val buffer = image.planes[0].buffer
            
            // Image Processing

            listeners.forEach { it() }

            image.close()
        }
    }
    ```

    
4. Create a `startCamera()` method to initialize the camera:

    ```kotlin
    override fun configureFlutterEngine(flutterEngine: FlutterEngine) {
        ...

        requestCameraPermission()

        channel = MethodChannel(flutterEngine.dartExecutor.binaryMessenger, CHANNEL)

        channel.setMethodCallHandler { call, result ->
            if (call.method == "startCamera") {
                startCamera(result)
            }
        }
    }

    private fun startCamera(result: MethodChannel.Result) {
        val cameraProviderFuture = ProcessCameraProvider.getInstance(this)
        cameraProviderFuture.addListener(
                {
                    val cameraProvider: ProcessCameraProvider = cameraProviderFuture.get()
                    bindCamera(cameraProvider, result)
                },
                ContextCompat.getMainExecutor(this)
        )
    }
    
    private fun bindCamera(provider: ProcessCameraProvider, result: MethodChannel.Result) {
        val metrics = windowManager.getCurrentWindowMetrics().bounds

        val screenAspectRatio = aspectRatio(metrics.width(), metrics.height())

        val rotation = display!!.rotation

        var resolutionSize = Size(previewWidth, previewHeight)

        if (rotation == ROTATION_0 || rotation == Surface.ROTATION_180) {
            resolutionSize = Size(previewHeight, previewWidth)
        }

        val cameraProvider =
                provider ?: throw IllegalStateException("Camera initialization failed.")

        val cameraSelector = CameraSelector.Builder().requireLensFacing(lensFacing).build()

        flutterTextureEntry = flutterEngine.renderer.createSurfaceTexture()

        // Preview
        preview =
                Preview.Builder()
                        .setTargetResolution(resolutionSize)
                         .setTargetRotation(rotation)
                        .build()
                        .also {
                            it.setSurfaceProvider { request ->
                                val surfaceTexture =
                                        flutterTextureEntry?.surfaceTexture().apply {
                                            this?.setDefaultBufferSize(request.resolution.width, request.resolution.height)
                                        }
                                val surface = Surface(surfaceTexture)

                                request.provideSurface(
                                        surface,
                                        ContextCompat.getMainExecutor(this)
                                ) {}
                            }
                        }

        imageAnalyzer =
                ImageAnalysis.Builder()
                        .setTargetResolution(resolutionSize)
                         .setTargetRotation(rotation)
                        .build()
                        .also {
                            it.setAnalyzer(
                                    cameraExecutor,
                                    ImageAnalyzer { results ->
                                        
                                    }
                            )
                        }

        if (camera != null) {
            removeCameraStateObservers(camera!!.cameraInfo)
        }

        try {
            cameraProvider.unbindAll() // Unbind use cases before rebinding
            camera = cameraProvider.bindToLifecycle(this, cameraSelector, preview, imageAnalyzer)
            observeCameraState(camera?.cameraInfo!!)
            result.success(flutterTextureEntry?.id())
        } catch (e: Exception) {
            result.error("CAMERA_INIT_FAILED", "Failed to initialize camera: ${e.message}", null)
        }
    }
    ```

    The `preview` and `imageAnalyzer` are created with the target resolution and rotation. The `setSurfaceProvider` method is used to provide the camera preview to the Flutter engine. The camera preview is displayed on the Flutter screen using the `flutterTextureEntry` ID.

5. Define methods `getPreviewWidth()` and `getPreviewHeight()` to fetch the camera preview resolution:

    ```kotlin
    private fun getPreviewWidth(): Double {
        return previewWidth.toDouble()
    }

    private fun getPreviewHeight(): Double {
        return previewHeight.toDouble()
    }
    ```

**CameraPreviewScreen.dart**

1. Initialize the camera preview:
    
    ```dart
    class _CameraPreviewScreenState extends State<CameraPreviewScreen> {
        static const platform = MethodChannel('barcode_scan');
        int? _textureId;
        double _previewWidth = 0.0;
        double _previewHeight = 0.0;
        bool isPortrait = false;
    
        @override
        void initState() {
            super.initState();
            platform.setMethodCallHandler(_handleMethod);
            _initializeCamera();
        }
    
        Future<void> _initializeCamera() async {
            try {
            final int textureId = await platform.invokeMethod('startCamera');
            final double previewWidth =
                await platform.invokeMethod('getPreviewWidth');
            final double previewHeight =
                await platform.invokeMethod('getPreviewHeight');
            setState(() {
                _textureId = textureId;
                _previewWidth = previewWidth;
                _previewHeight = previewHeight;
            });
            } catch (e) {
            print(e);
            }
        }
    }
    ```

2. Display the camera preview using the `Texture` widget:

    ```dart
    @override
    Widget build(BuildContext context) {
        final screenWidth = MediaQuery.of(context).size.width;
        final screenHeight = MediaQuery.of(context).size.height;
        var orientation = MediaQuery.of(context).orientation;
        isPortrait = orientation == Orientation.portrait;
        return Scaffold(
        body: _textureId == null
            ? const Center(child: CircularProgressIndicator())
            : SizedBox(
                width: screenWidth,
                height: screenHeight,
                child: FittedBox(
                    fit: BoxFit.cover,
                    child: Stack(
                    children: [
                        SizedBox(
                        width: isPortrait ? _previewHeight : _previewWidth,
                        height: isPortrait ? _previewWidth : _previewHeight,
                        child: Texture(textureId: _textureId!),
                        ),
                    ],
                    ),
                ),
                ),
        );
    }
    ```

    The `Texture` is wrapped in a `SizedBox` to maintain the correct aspect ratio of the camera preview. Without the `SizedBox`, the preview might be stretched or distorted. The `FittedBox` widget is used to scale the camera preview to fit the screen size. The `Stack` widget is used to overlay additional widgets on the camera preview. The `SizedBox` widget, set with screen width and height, is used to display the camera preview in full screen.

## Step 4: Integrating Dynamsoft Barcode Reader SDK
The Dynamsoft Barcode Reader SDK offers a straightforward API for decoding barcodes from images. Just a few lines of code can equip your Flutter application with robust barcode scanning capabilities.

**MainActivity.kt**

1. Activate the SDK with a valid license key:

    ```kotlin
    fun setLicense(license: String?) {
        BarcodeReader.initLicense(license) { isSuccessful, e ->
            if (isSuccessful) {
                // The license verification was successful.
            } else {
                // The license verification failed. e contains the error information.
            }
        }
    }

    override fun configureFlutterEngine(flutterEngine: FlutterEngine) {
        setLicens("LICENSE-KEY")
        ...
    }
    ```

2. Create a barcode reader instance in the `ImageAnalyzer` class:

    ```kotlin
    private class ImageAnalyzer(listener: ResultListener? = null) : ImageAnalysis.Analyzer {
        private val mBarcodeReader: BarcodeReader = BarcodeReader()

        ...
    }
    ```

3. Decode the barcode from the image and pass the results through listeners:

    ```kotlin
    typealias ResultListener = (results: List<Map<String, Any>>) -> Unit

    private class ImageAnalyzer(listener: ResultListener? = null) : ImageAnalysis.Analyzer {
        private val mBarcodeReader: BarcodeReader = BarcodeReader()
        private val listeners = ArrayList<ResultListener>().apply { listener?.let { add(it) } }

        private fun ByteBuffer.toByteArray(): ByteArray {
            rewind() 
            val data = ByteArray(remaining())
            get(data) 
            return data 
        }

        override fun analyze(image: ImageProxy) {
            if (listeners.isEmpty()) {
                image.close()
                return
            }

            val buffer = image.planes[0].buffer
            val stride = image.planes[0].rowStride

            val data = buffer.toByteArray()

            val results =
                    mBarcodeReader?.decodeBuffer(
                            data,
                            image.width,
                            image.height,
                            stride,
                            EnumImagePixelFormat.IPF_NV21
                    )
            listeners.forEach { it(wrapResults(results)) }

            image.close()
        }

        private fun wrapResults(results: Array<TextResult>?): List<Map<String, Any>> {
            val out = mutableListOf<Map<String, Any>>()
            if (results != null) {
                for (result in results) {
                    val data: MutableMap<String, Any> = HashMap()
                    data["format"] = result.barcodeFormatString
                    data["x1"] = result.localizationResult.resultPoints[0].x
                    data["y1"] = result.localizationResult.resultPoints[0].y
                    data["x2"] = result.localizationResult.resultPoints[1].x
                    data["y2"] = result.localizationResult.resultPoints[1].y
                    data["x3"] = result.localizationResult.resultPoints[2].x
                    data["y3"] = result.localizationResult.resultPoints[2].y
                    data["x4"] = result.localizationResult.resultPoints[3].x
                    data["y4"] = result.localizationResult.resultPoints[3].y
                    data["angle"] = result.localizationResult.angle
                    data["barcodeBytes"] = result.barcodeBytes
                    out.add(data)
                }
            }
            return out
        }
    }
    ```

4. Send the barcode results to the Dart side:

    ```kotlin
    imageAnalyzer =
                ImageAnalysis.Builder()
                        .setTargetResolution(resolutionSize)
                         .setTargetRotation(rotation)
                        .build()
                        .also {
                            it.setAnalyzer(
                                    cameraExecutor,
                                    ImageAnalyzer { results ->
                                        Handler(Looper.getMainLooper()).post {
                                            channel.invokeMethod("onBarcodeDetected", results)
                                        }
                                    }
                            )
                        }
    ```
    
    Note: The callback is executed on a background thread. To update the UI, use `Handler` to switch to the main thread.

**CameraPreviewScreen.dart**
1. Define a `BarcodeResult` class based on [https://github.com/yushulx/flutter_barcode_sdk/blob/main/lib/dynamsoft_barcode.dart](https://github.com/yushulx/flutter_barcode_sdk/blob/main/lib/dynamsoft_barcode.dart).
    
2. Retrieve and construct the barcode results. Rotate the coordinates of the barcode results by 90 degrees for portrait mode on Android:

    ```dart
    List<BarcodeResult> rotate90barcode(List<BarcodeResult> input, int height) {
      List<BarcodeResult> output = [];
      for (BarcodeResult result in input) {
        int x1 = result.x1;
        int x2 = result.x2;
        int x3 = result.x3;
        int x4 = result.x4;
        int y1 = result.y1;
        int y2 = result.y2;
        int y3 = result.y3;
        int y4 = result.y4;
    
        BarcodeResult newResult = BarcodeResult(
            result.format,
            result.text,
            height - y1,
            x1,
            height - y2,
            x2,
            height - y3,
            x3,
            height - y4,
            x4,
            result.angle,
            result.barcodeBytes);
    
        output.add(newResult);
      }
    
      return output;
    }

    Future<dynamic> _handleMethod(MethodCall call) async {
        if (call.method == "onBarcodeDetected") {
          barcodeResults =
              convertResults(List<Map<dynamic, dynamic>>.from(call.arguments));
          if (Platform.isAndroid && isPortrait && barcodeResults != null) {
            barcodeResults =
                rotate90barcode(barcodeResults!, _previewHeight.toInt());
          }
    
          setState(() {});
        }
      }
    ```

3. Draw the barcode results on the camera preview using the `CustomPainter` class:

    ```dart
    @override
    Widget build(BuildContext context) {
        final screenWidth = MediaQuery.of(context).size.width;
        final screenHeight = MediaQuery.of(context).size.height;
        var orientation = MediaQuery.of(context).orientation;
        isPortrait = orientation == Orientation.portrait;
        return Scaffold(
        body: _textureId == null
            ? const Center(child: CircularProgressIndicator())
            : SizedBox(
                width: screenWidth,
                height: screenHeight,
                child: FittedBox(
                    fit: BoxFit.cover,
                    child: Stack(
                    children: [
                        SizedBox(
                        width: isPortrait ? _previewHeight : _previewWidth,
                        height: isPortrait ? _previewWidth : _previewHeight,
                        child: Texture(textureId: _textureId!),
                        ),
                        Positioned(
                            left: 0,
                            top: 0,
                            right: 0,
                            bottom: 0,
                            child: createOverlay(
                            barcodeResults,
                            ))
                    ],
                    ),
                ),
                ),
        );
    }

    Widget createOverlay(
        List<BarcodeResult>? barcodeResults,
        ) {
        return CustomPaint(
            painter: OverlayPainter(barcodeResults),
        );
        }

        class OverlayPainter extends CustomPainter {
        List<BarcodeResult>? barcodeResults;

        OverlayPainter(this.barcodeResults);

        @override
        void paint(Canvas canvas, Size size) {
            final paint = Paint()
            ..color = Colors.blue
            ..strokeWidth = 5
            ..style = PaintingStyle.stroke;

            if (barcodeResults != null) {
            for (var result in barcodeResults!) {
                double minX = result.x1.toDouble();
                double minY = result.y1.toDouble();
                if (result.x2 < minX) minX = result.x2.toDouble();
                if (result.x3 < minX) minX = result.x3.toDouble();
                if (result.x4 < minX) minX = result.x4.toDouble();
                if (result.y2 < minY) minY = result.y2.toDouble();
                if (result.y3 < minY) minY = result.y3.toDouble();
                if (result.y4 < minY) minY = result.y4.toDouble();

                canvas.drawLine(Offset(result.x1.toDouble(), result.y1.toDouble()),
                    Offset(result.x2.toDouble(), result.y2.toDouble()), paint);
                canvas.drawLine(Offset(result.x2.toDouble(), result.y2.toDouble()),
                    Offset(result.x3.toDouble(), result.y3.toDouble()), paint);
                canvas.drawLine(Offset(result.x3.toDouble(), result.y3.toDouble()),
                    Offset(result.x4.toDouble(), result.y4.toDouble()), paint);
                canvas.drawLine(Offset(result.x4.toDouble(), result.y4.toDouble()),
                    Offset(result.x1.toDouble(), result.y1.toDouble()), paint);

                TextPainter textPainter = TextPainter(
                text: TextSpan(
                    text: result.text,
                    style: const TextStyle(
                    color: Colors.yellow,
                    fontSize: 22.0,
                    ),
                ),
                textAlign: TextAlign.center,
                textDirection: TextDirection.ltr,
                );
                textPainter.layout(minWidth: 0, maxWidth: size.width);
                textPainter.paint(canvas, Offset(minX, minY));
            }
            }
        }

        @override
        bool shouldRepaint(OverlayPainter oldDelegate) => true;
        }
    ```

    ![Flutter barcode reader](https://www.dynamsoft.com/codepool/img/2024/04/flutter-qr-code-scanner-android-camera.jpg)

## Source Code
[https://github.com/yushulx/flutter-barcode-scanner/tree/main/examples/native_camera](https://github.com/yushulx/flutter-barcode-scanner/tree/main/examples/native_camera)