Building an AR-Enhanced Pharma Lookup App with Flutter, Dynamsoft Barcode SDK and Database
Tags: flutter, webdev, programming, mobile
Published: 2023-11-30 02:46:35

An AR-integrated pharmaceutical lookup application is a specialized software tool that combines Augmented Reality (AR) technology with a pharmaceutical information database. This integration provides an interactive and enhanced method for accessing drug-related data. Users can scan pharmacy barcodes using their mobile devices to retrieve information about the medication, such as its ingredients, dosage, potential side effects, and other relevant details. In this article, we will develop a Flutter application that demonstrates the pharmaceutical lookup scenario, utilizing the Dynamsoft Barcode SDK and a synthetic dataset.

<video src="https://github.com/yushulx/pharma-lookup/assets/2202306/21a5c49d-840e-418b-b293-cb206cbbd894" controls="controls" muted="muted" style="max-height:640px; min-height: 200px; max-width: 100%;"></video>

## Try the Demo App Built with Flutter and Dynamsoft Barcode SDK
[https://yushulx.me/pharma-lookup/](https://yushulx.me/pharma-lookup/)

## Synthetic Dataset Generation and Deployment
We use **ChatGPT** to generate a synthetic dataset of pharmaceutical information and and upload it to a Google Sheets document. The dataset contains 10 rows of data, each including the lot number, drug name, manufacture date, expiration date, batch size, and quality check status.

![pharma dataset](https://www.dynamsoft.com/codepool/img/2023/11/pharma-dataset.png)

Navigate to the **Extensions** menu and select **Apps Script** to access the App Script editor. Once there, create a new script. 

![app script](https://www.dynamsoft.com/codepool/img/2023/11/app-script.png)

Copy and paste the following code into a new file or the existing default file within the script editor in Google Sheets.

```javascript
function doGet(e) {
  var records = [];
  var sheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
  var rows = sheet.getDataRange().getValues();

  rows.forEach(function(row, index) {
    if (index !== 0) { 
      var record = {}; 
      record.LotNumber = row[0];
      record.MedicationName = row[1];
      record.ManufactureDate = row[2];
      record.ExpirationDate = row[3];
      record.BatchSize = row[4];
      record.QualityCheckStatus = row[5];
      records.push(record);
    }
  });

  return ContentService.createTextOutput(JSON.stringify(records))
    .setMimeType(ContentService.MimeType.JSON);
}

```

The code above will convert the data in the Google Sheets document to a JSON object and return it as a response to the HTTP request.

Click on the **Deploy** button in the script editor, then select **New Deployment**. For the deployment type, choose **Web App**, and under the **Who has access** option, select **Anyone**. Finally, click **Deploy** to complete the deployment process.

![web app deployment](https://www.dynamsoft.com/codepool/img/2023/11/web-app-deployment.png)
 
You can now access the synthetic dataset through the provided URL to retrieve the JSON data. Try using the data source: [https://script.google.com/macros/s/AKfycbyPEx3THAbcLTNaJNOkQ1O3puTmQKXXOE_gkOGyKMzfIEUTr484qS8Dsi7-kTKpD333/exec](https://script.google.com/macros/s/AKfycbyPEx3THAbcLTNaJNOkQ1O3puTmQKXXOE_gkOGyKMzfIEUTr484qS8Dsi7-kTKpD333/exec).

With the lot numbers from the synthetic dataset, we can print out barcodes and label them on medical vials.

![medical vials](https://www.dynamsoft.com/codepool/img/2023/11/medical-vial.png)

## Real-time Pharmaceutical Lookup with Flutter Barcode Scanner
Since we have already implemented a Flutter barcode scanner app compatible with **Windows**, **Android**, **iOS** and **web**, we can repurpose the existing code and integrate new features. The source code is available on GitHub at: [https://github.com/yushulx/flutter-barcode-scanner](https://github.com/yushulx/flutter-barcode-scanner)

To implement the AR-enhanced pharmaceutical lookup app, follow these two steps:
1. Fetch the pharmaceutical information from Google Sheets and store it in a global variable.
2. Match the scanned barcode with the corresponding lot number and display the pharmaceutical information.

### Fetch the Dataset from Google Sheets
1. Add the `http` package to the `pubspec.yaml` file for handling HTTP requests.
    ```yaml
    dependencies:
      flutter:
        sdk: flutter
      http: ^1.1.2
    ```
2. In the `main.dart` file, create a `_fetchData()` function that makes a `GET HTTP` request to fetch the JSON data.
    ```dart
    Future<void> _fetchData() async {
        final response = await http.get(Uri.parse(
            'https://script.google.com/macros/s/AKfycbyPEx3THAbcLTNaJNOkQ1O3puTmQKXXOE_gkOGyKMzfIEUTr484qS8Dsi7-kTKpD333/exec'));
    
        if (response.statusCode == 200) {
          List<dynamic> list = json.decode(response.body);
          database = {
            for (var item in list)
              item['LotNumber'] as String: Pharma.fromJson(item)
          };
        } else {
          throw Exception('Failed to load data');
        }
      }
    ```

    The `Pharma` class is defined as follows:
    ```dart
    class Pharma {
      final String lotNumber;
      final String medicationName;
      final DateTime manufactureDate;
      final DateTime expirationDate;
      final int batchSize;
      final String qualityCheckStatus;
    
      Pharma({
        required this.lotNumber,
        required this.medicationName,
        required this.manufactureDate,
        required this.expirationDate,
        required this.batchSize,
        required this.qualityCheckStatus,
      });
    
      factory Pharma.fromJson(Map<String, dynamic> json) {
        return Pharma(
          lotNumber: json['LotNumber'] as String,
          medicationName: json['MedicationName'] as String,
          manufactureDate: DateTime.parse(json['ManufactureDate']),
          expirationDate: DateTime.parse(json['ExpirationDate']),
          batchSize: int.parse(json['BatchSize'].toString()),
          qualityCheckStatus: json['QualityCheckStatus'] as String,
        );
      }
    }
    ```

    The JSON data will be converted to a `Map` object, with the lot number as the key and the `Pharma` object as the value.

    ```dart
    Map<String, Pharma> database = {};
    ```

### Match the Barcode with the Lot Number and Display the Pharmaceutical Information

In the `global.dart` file, locate the `paint()` function, and then add the provided code within this function to render the pharmaceutical information on the screen.

```dart
if (database.containsKey(result.text)) {
  Pharma pharma = database[result.text]!;
    // Draw the pharmaceutical information
    // Background
    // Medication Name
    // Manufacture Date
    // Expiration Date
    // Batch Size
    // Quality Check Status
}
```

- Background:
    
    ```dart
    final bgPaint1 = Paint()
            ..color = Colors.green.withOpacity(0.6)
            ..style = PaintingStyle.fill;
    const double bgWidth = 220, bgHeight = 110;

    minY = minY - bgHeight - 10;
    var backgroundRect = Rect.fromLTWH(minX, minY + bgHeight, bgWidth, 5);
    canvas.drawRect(backgroundRect, bgPaint1);

    final bgPaint2 = Paint()
      ..color = Colors.black.withOpacity(0.6)
      ..style = PaintingStyle.fill;
    backgroundRect = Rect.fromLTWH(minX, minY, bgWidth, bgHeight);
    canvas.drawRect(backgroundRect, bgPaint2);
    ```
- Medication Name:
    
    ```dart
    const int xSpacing = 100, ySpacing = 20, padding = 10;
    const nameSpan = TextSpan(
      style: TextStyle(
          color: Colors.green, fontSize: 14, fontWeight: FontWeight.bold),
      text: 'Name',
    );
    final namePainter = TextPainter(
      text: nameSpan,
      textDirection: TextDirection.ltr,
    );
    namePainter.layout(minWidth: 0, maxWidth: size.width);
    namePainter.paint(canvas, Offset(padding + minX, padding + minY));

    var nameValue = TextSpan(
      style: const TextStyle(
          color: Colors.white, fontSize: 14, fontWeight: FontWeight.bold),
      text: pharma.medicationName,
    );
    final nameValuePainter = TextPainter(
      text: nameValue,
      textDirection: TextDirection.ltr,
    );
    nameValuePainter.layout(minWidth: 0, maxWidth: size.width);
    nameValuePainter.paint(
        canvas,
        Offset(padding + minX + xSpacing,
            padding + minY));
    ```
- Manufacture Date:
    
    ```dart
    const manufactureDateSpan = TextSpan(
            style: TextStyle(color: Colors.green, fontSize: 12),
            text: 'Manufacture Date',
          );
    final manufactureDatePainter = TextPainter(
      text: manufactureDateSpan,
      textDirection: TextDirection.ltr,
    );
    manufactureDatePainter.layout(minWidth: 0, maxWidth: size.width);
    manufactureDatePainter.paint(
        canvas,
        Offset(padding + minX,
            minY + ySpacing + padding)); 

    var manufactureDateValueSpan = TextSpan(
      style: const TextStyle(color: Colors.white, fontSize: 12),
      text:
          '${pharma.manufactureDate.day}/${pharma.manufactureDate.month}/${pharma.manufactureDate.year}',
    );
    final manufactureDateValuePainter = TextPainter(
      text: manufactureDateValueSpan,
      textDirection: TextDirection.ltr,
    );
    manufactureDateValuePainter.layout(minWidth: 0, maxWidth: size.width);
    manufactureDateValuePainter.paint(canvas,
        Offset(padding + minX + xSpacing, minY + ySpacing + padding));
    ```
- Expiration Date:
    
     ```dart
    const expirationDateSpan = TextSpan(
            style: TextStyle(color: Colors.green, fontSize: 12),
            text: 'Expiration Date',
          );

    final expirationDatePainter = TextPainter(
      text: expirationDateSpan,
      textDirection: TextDirection.ltr,
    );

    expirationDatePainter.layout(minWidth: 0, maxWidth: size.width);
    expirationDatePainter.paint(
        canvas,
        Offset(
            padding + minX,
            minY +
                2 * ySpacing +
                padding)); 

    var expirationDateValueSpan = TextSpan(
        style: const TextStyle(color: Colors.white, fontSize: 12),
        text:
            '${pharma.expirationDate.day}/${pharma.expirationDate.month}/${pharma.expirationDate.year}');

    final expirationDateValuePainter = TextPainter(
      text: expirationDateValueSpan,
      textDirection: TextDirection.ltr,
    );

    expirationDateValuePainter.layout(minWidth: 0, maxWidth: size.width);
    expirationDateValuePainter.paint(canvas,
        Offset(padding + minX + xSpacing, minY + 2 * ySpacing + padding));

    ```

- Batch Size:
    
    ```dart
    const batchSizeSpan = TextSpan(
            style: TextStyle(color: Colors.green, fontSize: 12),
            text: 'Batch Size',
          );

    final batchSizePainter = TextPainter(
      text: batchSizeSpan,
      textDirection: TextDirection.ltr,
    );

    batchSizePainter.layout(minWidth: 0, maxWidth: size.width);
    batchSizePainter.paint(
        canvas,
        Offset(
            padding + minX,
            minY +
                3 * ySpacing +
                padding)); 

    var batchSizeValueSpan = TextSpan(
        style: const TextStyle(color: Colors.white, fontSize: 12),
        text: pharma.batchSize.toString());

    final batchSizeValuePainter = TextPainter(
      text: batchSizeValueSpan,
      textDirection: TextDirection.ltr,
    );

    batchSizeValuePainter.layout(minWidth: 0, maxWidth: size.width);
    batchSizeValuePainter.paint(canvas,
        Offset(padding + minX + xSpacing, minY + 3 * ySpacing + padding));
    ```
- Quality Check Status:
    
    ```dart
    const qualityCheckStatusSpan = TextSpan(
            style: TextStyle(color: Colors.green, fontSize: 12),
            text: 'Quality Status',
          );

    final qualityCheckStatusPainter = TextPainter(
      text: qualityCheckStatusSpan,
      textDirection: TextDirection.ltr,
    );

    qualityCheckStatusPainter.layout(minWidth: 0, maxWidth: size.width);
    qualityCheckStatusPainter.paint(
        canvas,
        Offset(
            padding + minX,
            minY +
                4 * ySpacing +
                padding)); 

    var qualityCheckStatusValueSpan = TextSpan(
        style: pharma.qualityCheckStatus == 'Passed'
            ? const TextStyle(color: Colors.green, fontSize: 12)
            : const TextStyle(color: Colors.red, fontSize: 12),
        text: pharma.qualityCheckStatus);

    final qualityCheckStatusValuePainter = TextPainter(
      text: qualityCheckStatusValueSpan,
      textDirection: TextDirection.ltr,
    );

    qualityCheckStatusValuePainter.layout(
        minWidth: 0, maxWidth: size.width);
    qualityCheckStatusValuePainter.paint(canvas,
        Offset(padding + minX + xSpacing, minY + 4 * ySpacing + padding));
    ```

### Run the App and Scan the Barcode on the Medical Vial
```bash
flutter run
# flutter run -d windows
# flutter run -d edge
```
    
![pharma lookup](https://www.dynamsoft.com/codepool/img/2023/11/pharma-look-up.png)

## Source Code
[https://github.com/yushulx/pharma-lookup](https://github.com/yushulx/pharma-lookup)