[Huawei Developers] Freemind R:
This article is originally from HUAWEI Developer Forum

Forum link: https://forums.developer.huawei.com/forumPortal/en/home​

Introduction

Huawei HMS MLKit provides more and more services. In the last issue, we introduced the bank card identification function. A few lines of codes can recognize bank card numbers if using dedicated bank card identification interface. Some partners may ask the following questions:

I want to identify membership card rather than bank card. Can card number be identified?

The answer is yes. There is no problem. However, because the card number locations and layouts of different cards are different, the dedicated bank card identification API cannot be used like bank card identification. In this case, the general card identification capability can provide a standard general card alignment box, you only need to pull up the alignment box to obtain all the contents of the card. Then, you can extract key information based on the contents according to certain rules to obtain the desired contents such like eight consecutive digits are the card number, or the contents after some special letters are the card number.

Scenarios of General Card OCR

General card identification can identify a wide range of cards.

Card recognition for passenger info

Travel apps need to be bound to information such as the ID card number for ticket purchase and hotel reservation. The system automatically identifies the ID card number by taking photos, preventing manual input of the ID card number.

Shopping card binding

A shopping app that takes photos to identify membership cards, shopping cards, and discount cards.

[IMG="https://communityfile-dre.op.hicloud.com/FileServer/getFile/cmtybbs/034/238/374/5190001000034238374.20200523093406.68600850292976569790150510275474:50510603064639:2800:FE6E0B160085A4B09B917C451A8CB58113A7F5AF3965BCE3A2DDD3A1BC9CF27E.png"]https://communityfile-dre.op.hicloud.com/FileServer/getFile/cmtybbs/034/238/374/5190001000034238374.20200523093406.68600850292976569790150510275474:50510603064639:2800:FE6E0B160085A4B09B917C451A8CB58113A7F5AF3965BCE3A2DDD3A1BC9CF27E.png[/IMG]

Although bank cards and China's second-generation ID cards can be recognized by general cards, it is recommended that the dedicated bank card recognition and ID card recognition APIs of Huawei ML Kit be used. Dedicated card recognition is optimized for bank cards and ID cards, and has higher accuracy, the processed card number data can be directly returned. The differences between the private card and the general card can be described in detail later.

Tips: All device capabilities are free of charge, and all android devices are covered------both for Huawei phones and Non-Huawei phones.

Development process

This article will introduce u how to build up card recognition capability for Hong Kong and Macao Permit, Home Visit Permit, and Hong Kong ID Card

1 Preparation

For more details, please infer to official website：

https://developer.huawei.com/consumer/en/doc/development/HMS-Guides/ml-process-4

Here is the key steps:

1.1 Adding Huawei maven in project level gradle

Open the AndroidStudio project-level build.gradle file and incrementally add the following Maven address:

buildscript {
repositories {
maven {url 'http://developer.huawei.com/repo/'}
} }allprojects {
repositories {
maven { url 'http://developer.huawei.com/repo/'}
}}

1.2 Add the SDK dependency to build.gradle at the application level.
dependencies{
// introduce base SDK
implementation 'com.huawei.hms:ml-computer-vision-ocr:1.0.3.300'
// introduce Latin character model
implementation 'com.huawei.hms:ml-computer-vision-ocr-latin-model:1.0.3.300'
// introduce bank card recognition plugin
implementation 'com.huawei.hms:ml-computer-card-gcr-plugin:1.0.3.300' }

Adding following sentences into AndroidManifest.xml:
<manifest
...
<meta-data
android:name="com.huawei.hms.ml.DEPENDENCY"
android:value= "ocr"/>
... </manifest>

1.2 Add the SDK dependency to build.gradle at the application level.

dependencies{
// introduce base SDK
implementation 'com.huawei.hms:ml-computer-vision-ocr:1.0.3.300'
// introduce Latin character model
implementation 'com.huawei.hms:ml-computer-vision-ocr-latin-model:1.0.3.300'
// introduce bank card recognition plugin
implementation 'com.huawei.hms:ml-computer-card-gcr-plugin:1.0.3.300' }

Adding following sentences into AndroidManifest.xml:

<manifest
...
<meta-data
android:name="com.huawei.hms.ml.DEPENDENCY"
android:value= "ocr"/>
... </manifest>

1.3 Configure obfuscation script

Please infer to following link: https://developer.huawei.com/consumer/cn/doc/development/HMS-Guides/ml-assigning-permissions-4

2 development

2.1 card identification

@Override
public void onClick(View v) {
switch (v.getId()) {
// Button for detecting album pictures.
case R.id.detect_picture:
this.startLocalImageActivity(cardImage, null, callback);
break;
// 视频流检测按钮。
case R.id.detect_video:
this.startCaptureActivity(null, callback);
break;
// button for detecting
case R.id.detect_take_photo:
this.startTakePhotoActivity(null, callback);
break;
default:
break;
} }

video from camera source

private void startCaptureActivity(Object object, MLGcrCapture.Callback callback) {
// create general card analyzer
MLGcrCaptureConfig cardConfig = new MLGcrCaptureConfig.Factory().create();
MLGcrCaptureUIConfig uiConfig = new MLGcrCaptureUIConfig.Factory()
.setScanBoxCornerColor(Color.GREEN)
// Set the text in the scan box. It is recommended that the text contain less than 30 characters.
.setTipText("Recognizing, align edges")
// MLGcrCaptureUIConfig.ORIENTATION_AUTO: Automatic mode. The display direction is determined by the physical sensor.
// MLGcrCaptureUIConfig.ORIENTATION_LANDSCAPE: Landscape mode.
// MLGcrCaptureUIConfig.ORIENTATION_PORTRAIT: Portrait mode.
.setOrientation(MLGcrCaptureUIConfig.ORIENTATION_AUTO)
.create();
// Method 1: Create a general-purpose card identification processor based on the customized card identification interface configurator.
MLGcrCapture ocrManager = MLGcrCaptureFactory.getInstance().getGcrCapture(cardConfig, uiConfig);
// Method 2: Use the default interface to create a general-purpose card identification processor.
MLGcrCapture ocrManager = MLGcrCaptureFactory.getInstance().getGcrCapture(cardConfig);
ocrManager.capturePreview(this, object, callback); }

photo from camera source

private void startTakePhotoActivity(Object object, MLGcrCapture.Callback callback) {
MLGcrCaptureConfig cardConfig = new MLGcrCaptureConfig.Factory().create();
// Create a general card recognition configurator
MLGcrCaptureUIConfig uiConfig = new MLGcrCaptureUIConfig.Factory()
// Sets the color of the scan box.
.setScanBoxCornerColor(Color.BLUE)
// Set the text in the scan box. It is recommended that the text contain less than 30 characters.
.setTipText("Taking picture, align edges")
// The landscape and portrait modes are as follows:
// MLGcrCaptureUIConfig.ORIENTATION_AUTO: Automatic mode. The display direction is determined by the physical sensor. // MLGcrCaptureUIConfig.ORIENTATION_LANDSCAPE
// MLGcrCaptureUIConfig.ORIENTATION_PORTRAIT
.setOrientation(MLGcrCaptureUIConfig.ORIENTATION_AUTO)
.create();
// Method 1: Create a general-purpose card identification processor based on the customized card identification interface configurator.
MLGcrCapture ocrManager = MLGcrCaptureFactory.getInstance().getGcrCapture(cardConfig, uiConfig);
// Method 2: Use the default interface to create a general-purpose card identification processor.
MLGcrCapture ocrManager = MLGcrCaptureFactory.getInstance().getGcrCapture(cardConfig);
// Bind the general card identification processor and the processing result callback function.
ocrManager.capturePhoto(this, object, callback); }

photo from local

private void startLocalImageActivity(Bitmap bitmap, Object object, MLGcrCapture.Callback callback) {
// create general card analyzer
MLGcrCaptureConfig config = new MLGcrCaptureConfig.Factory().create();
MLGcrCapture ocrManager = MLGcrCaptureFactory.getInstance().getGcrCapture(config);
// The bitmap is the bitmap image to be identified. The following image formats are supported:jpg/jpeg/png/bmp。
ocrManager.captureImage(bitmap, object, callback); }

2.2 Perform post-processing on the identified content and extract key information.

Reload the onResult, onCanceled, onFailure, and onDenied methods. OnResult indicates that the result is returned. MLGcrCaptureResult indicates the result returned by card identification. onCanceled indicates that the user cancels the card identification. onFailure indicates that the identification fails. onDenied indicates that the camera is unavailable.

private MLGcrCapture.Callback callback = new MLGcrCapture.Callback() {
@Override
public int onResult(MLGcrCaptureResult result, Object object) {
Log.i(TAG, "callback onRecSuccess");
if (result == null) {
Log.e(TAG, "callback onRecSuccess result is null");
return MLGcrCaptureResult.CAPTURE_CONTINUE;
}

GeneralCardProcessor idCard = null;
GeneralCardResult cardResult = null;
if (cardTypeEnum == CardType.PASSCARD) {
idCard = new PassCardProcessor(result.text);
} else if (cardTypeEnum == CardType.HKIDCARD) {
idCard = new HKIdCardProcessor(result.text);
} else if (cardTypeEnum == CardType.COMEHOMECARD) {
idCard = new HomeCardProcessor(result.text);
}
if (idCard != null) {
cardResult = idCard.getResult();
}

showFrontImage(result.cardBitmap);
displayResult(cardResult);

// If the results don't match
if (cardResult == null || cardResult.valid.isEmpty() || cardResult.number.isEmpty()) {
return MLGcrCaptureResult.CAPTURE_CONTINUE;
}

displayResult(cardResult);
return MLGcrCaptureResult.CAPTURE_STOP;
}
};}
};v

The card number extraction logic can be implemented by rewriting the getResult() method in the GeneralCardProcessor class. The following uses the Hong Kong, Macao, and Taiwan passes as an example. For details, see the source code on GitHub.

public class PassCardProcessor implements GeneralCardProcessor {
private static final String TAG = "PassCardProcessor";

private final MLText text;

public PassCardProcessor(MLText text) {
this.text = text;
}
@Override
public GeneralCardResult getResult() {
List<MLText.Block> blocks = text.getBlocks();
if (blocks.isEmpty()) {
Log.i(TAG, "Result blocks is empty");
return null;
}
ArrayList<BlockItem> originItems = getOriginItems(blocks);
String valid = "";
String number = "";
boolean validFlag = false;
boolean numberFlag = false;
for (BlockItem item : originItems) {
String tempStr = item.text;
if (!validFlag) {
String result = tryGetValidDate(tempStr);
if (!result.isEmpty()) {
valid = result;
validFlag = true;
}
}
if (!numberFlag) {
String result = tryGetCardNumber(tempStr);
if (!result.isEmpty()) {
number = result;
numberFlag = true;
}
}
}
return new GeneralCardResult(valid, number);
} }

Demo

[IMG="https://communityfile-dre.op.hicloud.com/FileServer/getFile/cmtybbs/034/238/374/5190001000034238374.20200523094122.65931725678037125224758299597386:50510603064639:2800:B87F6D0CF76FAB1B2C783AB787743D0CD2B77DD7AB57F5359E2C12CA1CCDFA94.gif"]https://communityfile-dre.op.hicloud.com/FileServer/getFile/cmtybbs/034/238/374/5190001000034238374.20200523094122.65931725678037125224758299597386:50510603064639:2800:B87F6D0CF76FAB1B2C783AB787743D0CD2B77DD7AB57F5359E2C12CA1CCDFA94.gif[/IMG]

GitHub source code

The source code has been uploaded to GitHub. You can improve this function on it.

GitHub source code address:

https://github.com/HMS-MLKit/HUAWEI-HMS-MLKit-Sample

Path for storing the demo code of General Card OCR:

MLKit-Sample\module-text\src\main\java\com\mlkit\sample\activity\GeneralCardRecognitionActivity.java