How to Recognize Rotated Text with OCR and Barcode SDK
Published: 2021-03-26 06:17:39

The accuracy of optical character recognition (OCR) is sensitive to text orientation. If a text image file is rotated by 90°, 180° or 270°, and there is no EXIF information, it would be challenging to doing text recognition. In case of text around barcode, the combination of Dynamsoft Label Recognition(DLR) and Dynamsoft Barcode Reader(DBR) is proposed as an ideal OCR solution. This article will demonstrate how to build a C++ command-line application to recognize a rotated barcode label (e.g. price tag) which is hardly recognized by general open source or commercial OCR SDK.

![OCR: rotated text recognition](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/euobkk7fp73hwva1k91p.png)

## Requirements
- [Dynamsoft Barcode Reader SDK](https://www.dynamsoft.com/barcode-reader/downloads) and [DBR 30-day FREE Trial License](https://www.dynamsoft.com/customer/license/trialLicense/?product=dbr)
- [Dynamsoft Label Recognition SDK](https://www.dynamsoft.com/label-recognition/downloads) and [DLR 30-day FREE Trial License](https://www.dynamsoft.com/customer/license/trialLicense/?product=dlr)
- [OpenCV 4.5.0](https://opencv.org/releases/)
- [CMake](https://cmake.org/download/)

## DLR API Documentation
https://www.dynamsoft.com/label-recognition/programming/c-cplusplus/user-guide.html?ver=latest

## A New OCR Approach: Text Recognition by Barcode Orientation Detection
Both Dynamsoft Barcode SDK and Dynamsoft OCR SDK support JSON-formatted template import. To parse JSON in C++, we use [json.hpp](https://github.com/nlohmann/json), which is a single header file for easy integration. No dependencies required.


A basic template file for DLR runtime settings taking advantage of barcode results is as follows:

```json
{
   "CharacterModelArray" : [
      {
         "Name" : "NumberLetter"
      }
   ],
   "LabelRecognitionParameterArray" : [
      {
         "Name" : "BarcodeLabel",
         "ReferenceRegionNameArray" : [ "defaultReferenceRegion" ]
      }
   ],
   "ReferenceRegionArray" : [
      {
         "Localization" : {
            "BarcodeFormatIds" : [ "DLR_BF_ALL" ]
         },
         "Name" : "defaultReferenceRegion"
      }
   ]
  
}

```

The barcode zone is used as the reference region for text recognition. By default, the text searching area is under the barcode.

The `BarcodeFormatIds` is mandatory. If the input barcode symbologies are not fixed, we set `DLR_BF_ALL` for covering primary 1D and 2D barcode symbologies. 

Let's create DLR and DBR objects and construct a JSON object by loading the template file:

```cpp
#include <string>
#include <fstream>
#include <iostream>

#include "include/DynamsoftLabelRecognition.h"
#include "include/DynamsoftCommon.h"
#include "include/DynamsoftBarcodeReader.h"
#include "include/json.hpp"

using json = nlohmann::json;
using namespace dynamsoft::dlr;
using namespace dynamsoft::dbr;

CLabelRecognition dlr;
dlr.InitLicense("LICENSE-KEY");

CBarcodeReader dbr;
dbr.InitLicense("LICENSE-KEY");

std::ifstream templateFile("<template file>");
json templateObj;
templateFile >> templateObj;
```

Before invoking the OCR APIs, we determine the reference regions based on barcode recognition results:

```cpp
TextResultArray *resultArray = NULL;
dbr.DecodeBuffer(imageCopy.data, imgWidth, imgHeight, imageCopy.step.p[0], IPF_RGB_888,"");
dbr.GetAllTextResults(&resultArray);

dlr.ResetRuntimeSettings();
DLRRuntimeSettings settings = {};
dlr.GetRuntimeSettings(&settings);
string templateName = templateObj["LabelRecognitionParameterArray"][0]["Name"];
dlr.AppendSettingsFromString(templateObj.dump().c_str());
dlr.UpdateReferenceRegionFromBarcodeResults(resultArray, templateName.c_str());
dlr.UpdateRuntimeSettings(&settings);
```

Here are the decoding APIs used for different data source types:

- [DecodeFile](https://www.dynamsoft.com/barcode-reader/programming/cplusplus/api-reference/cbarcodereader-methods/decode.html#decodefile)
- [DecodeFileInMemory](https://www.dynamsoft.com/barcode-reader/programming/cplusplus/api-reference/cbarcodereader-methods/decode.html#decodefileinmemory)
- [DecodeBuffer](https://www.dynamsoft.com/barcode-reader/programming/cplusplus/api-reference/cbarcodereader-methods/decode.html#decodebuffer)
- [DecodeBase64String](https://www.dynamsoft.com/barcode-reader/programming/cplusplus/api-reference/cbarcodereader-methods/decode.html#decodebase64string)
- [DecodeDIB](https://www.dynamsoft.com/barcode-reader/programming/cplusplus/api-reference/cbarcodereader-methods/decode.html#decodedib)

Afterward, we can either use [RecognizeByBuffer](https://www.dynamsoft.com/label-recognition/programming/c-cplusplus/api-reference/c-label-recognition-class/recognizing.html#recognizebybuffer) or [RecognizeByFile](https://www.dynamsoft.com/label-recognition/programming/c-cplusplus/api-reference/c-label-recognition-class/recognizing.html#recognizebyfile) to recognize text:

```cpp
// buffer
DLRImageData data = {imageCopy.step.p[0] * imgHeight, imageCopy.data, imgWidth, imgHeight, imageCopy.step.p[0], DLR_IPF_RGB_888};
dlr.RecognizeByBuffer(&data, templateName.c_str());

// file
dlr.RecognizeByFile("<image file>", templateName.c_str());
```

The returned result is an array, from which we can get text and corresponding coordinate values:

```cpp
DLRResultArray* pDLRResults = NULL;
dlr.GetAllDLRResults(&pDLRResults);
if (pDLRResults != NULL)
{
  int rCount = pDLRResults->resultsCount;
  for (int ri = 0; ri < rCount; ++ri)
  {
    DLRResult* result = pDLRResults->results[ri];
    int lCount = result->lineResultsCount;

    for (int li = 0; li < lCount; ++li)
    {
      printf("Line result %d: %s\r\n", li, result->lineResults[li]->text);
      DLRPoint *points = result->lineResults[li]->location.points;
      printf("x1: %d, y1: %d, x2: %d, y2: %d, x3: %d, y3: %d, x4: %d, y4: %d\r\n", points[0].x, 
      points[0].y, points[1].x, points[1].y, points[2].x, points[2].y, points[3].x, points[3].y);
    }
  }
}
dlr.FreeDLRResults(&pDLRResults);
if (resultArray) CBarcodeReader::FreeTextResults(&resultArray);
```

### Label OCR Test by Orientation
To prove the efficiency of combining Dynamsoft barcode and OCR SDKs, we can make a simple experiment by rotating a barcode label image.

**Sample Image**
![codabar](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/16cco8f1tkh14lkstew1.jpg)

**0 degree**

![text recognition by 0 degree](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qvkviovcu33nqq4f4tni.png)

**90 degree**

![text recognition by 90 degree](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oispqd21wbi9njciuie4.png)


**180 degree** 

![text recognition by 180 degree](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kiby56roq3q8c6wpn6pu.png)

**270 degree**

![text recognition by 270 degree](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4xugojwff2is0y8qgjww.png)


As long as there is a barcode exists, the text aside can be successfully recognized.

## Source Code
[https://github.com/Dynamsoft/OCR-barcode-label](https://github.com/Dynamsoft/OCR-barcode-label)