[Huawei Developers] John Wo:
Introducton:

HUAWEI HiAI is an open Artificial Intelligence (AI) capable platform for smart devices, which adopts a chip-device-cloud architecture, opening up the chip, app, and service capabilities for a fully intelligent ecosystem. The HiAI is used for Text Recognition, Image Recognition etc. Text Recognition is used to extract text data from an image and the Image recognition is used to obtain quality, category and scene of a particular image.

This article is giving a brief explanation on Document Skew Correction, General Text Recognition and Table Recognition. Here we are using DevEco plugin to configure the HiAI application. To know about the integrate of application via DevEco you can refer the article "HUAWEI HiAI Image Super-Resolution Via DevEco".

General Text Recognition:

The API detects and identifies text information in a given image. It supports text extraction from screenshots with simple to more complex backgrounds. In snapshot Optical Character Recognition(OCR), the image resolution should be higher than 720p (1280×720 px), and the aspect ratio (length-to-width ratio) should be lower than 2:1. If the input image does not meet the above requirements, text recognition accuracy may be affected.

Code Implementation:

Initialize with the VisionBase static class and asynchronously get the connection of the service.

private void initHiAI() {
VisionBase.init(this, new ConnectionCallback() {
@Override
public void onServiceConnect() {
/** Invoke the callback method when the service is connected successfully. You can also perform initialization and mark the service connection status in this method*/
}

@Override
public void onServiceDisconnect() {
/** Invoke the callback method when the service is disconnected. You can choose to reconnect to the service or handle exceptions API invocation*/
}
});

}

Define the detector instance and frame. Select the OCR type to be called via TextConfiguration and finally process the Json result.

private void setHiAi() {

/** Define the detector instance and use the Context of this project as a parameter*/
TextDetector detector = new TextDetector(this);
/** Define a frame and initialize it*/
Frame frame = new Frame();

/** Place the bitmap to be checked in the frame*/
frame.setBitmap(bitmap);
/** Select the OCR type to be called via TextConfiguration, for example TYPE_TEXT_DETECT_SCREEN_SHOT_GENERAL is the detection screenshot*/
TextConfiguration config = new TextConfiguration();
config.setEngineType(TextDetectType.TYPE_TEXT_DETECT_SCREEN_SHOT_GENERAL);
detector.setTextConfiguration(config);
/** Check the text and process the result. Note that this line of code must be placed in the working thread rather than the primary thread*/
JSONObject jsonObject = detector.detect(frame, null);
/** Run the convertResult() command to convert the JSON character string into a Java class. You can also parse the JSON character string yourself*/
Text text = detector.convertResult(jsonObject);
System.out.println("Json:"+jsonObject);
String txtValue = text.getValue();
this.txtValue = txtValue;
handler.sendEmptyMessage(Utils.HiAI_RESULT);
}

Screenshot:

[IMG="https://communityfile-dre.op.hicloud.com/FileServer/getFile/cmtybbs/001/745/226/2640091000001745226.20201030115254.25768642126949978549871058748230:50511111010457:2800:B276D6D761BC00193D54640FABA6F57BE945FA5B056EC870AAC02C1AFB9B3BF1.png"]https://communityfile-dre.op.hicloud.com/FileServer/getFile/cmtybbs/001/745/226/2640091000001745226.20201030115254.25768642126949978549871058748230:50511111010457:2800:B276D6D761BC00193D54640FABA6F57BE945FA5B056EC870AAC02C1AFB9B3BF1.png[/IMG]

[IMG="https://communityfile-dre.op.hicloud.com/FileServer/getFile/cmtybbs/001/745/226/2640091000001745226.20201030115310.17801746822842847194551705222544:50511111010457:2800:EC7582B702E68F4B5EDF20ECD52F4160844C8FAD932BA19C87313E96F5477F28.png"]https://communityfile-dre.op.hicloud.com/FileServer/getFile/cmtybbs/001/745/226/2640091000001745226.20201030115310.17801746822842847194551705222544:50511111010457:2800:EC7582B702E68F4B5EDF20ECD52F4160844C8FAD932BA19C87313E96F5477F28.png[/IMG]

Document Skew Correction:

The API can be used to recognize automatically the location information of the document in the image and correct the angle of the document. For example, we can duplicate paper photos and letters and correct their locations, thereby reserving memories.

Here the bitmap of the frame type must be in the ARGB8888 format with a size not greater than 20 megapixels.

Code Implementation:

As mentioned above sample, we have to Initialize with the VisionBase static class. After that define detector instance and frame. Put the picture which need to be detected into the frame. Call the detect method and process the result.

private void setHiAi() {
/** Define class detector, the context of this project is the input parameter */
DocRefine docResolution = new DocRefine(this);
/** Get Bitmap image (Note: bitmap type must be ARGB8888, please pay attention to the necessary conversion)*/

/** Define frame class, and put the picture which need to be detected into the frame */
Frame frame = new Frame();
frame.setBitmap(bitmap);
/** Call the detect method to get the information of the result*/
/** Note: This line of code must be placed in the worker thread instead of the main thread */
JSONObject jsonDoc = docResolution.docDetect(frame, null);
/** Returns a jsonObject format, you can call convertResult() method to convert the json to java class and get the result*/
DocCoordinates sc = docResolution.convertResult(jsonDoc);
ImageResult sr = docResolution.docRefine(frame, sc, null);
/** Get bitmap result*/
Bitmap bmp = sr.getBitmap();

/** Note: The result and the Bitmap in the result must be NULL, but also to determine whether the returned error code is 0 (0 means no error)*/
this.bmp = bmp;
handler.sendEmptyMessage(Utils.HiAI_RESULT);
}

Screenshot:

[IMG="https://communityfile-dre.op.hicloud.com/FileServer/getFile/cmtybbs/001/745/226/2640091000001745226.20201030115344.75799732364770329042915870510471:50511111010457:2800:FFD948F29F3EB521F7FCE42846B811313797AD30E1AE8451907DB9FF46653FDA.png"]https://communityfile-dre.op.hicloud.com/FileServer/getFile/cmtybbs/001/745/226/2640091000001745226.20201030115344.75799732364770329042915870510471:50511111010457:2800:FFD948F29F3EB521F7FCE42846B811313797AD30E1AE8451907DB9FF46653FDA.png[/IMG]

[IMG="https://communityfile-dre.op.hicloud.com/FileServer/getFile/cmtybbs/001/745/226/2640091000001745226.20201030115355.90018812233408244416354461612421:50511111010457:2800:BE68F761D0043102627C0C287F285B9B1D0D28A9B8DD6A01825728BEDFF0EAAE.png"]https://communityfile-dre.op.hicloud.com/FileServer/getFile/cmtybbs/001/745/226/2640091000001745226.20201030115355.90018812233408244416354461612421:50511111010457:2800:BE68F761D0043102627C0C287F285B9B1D0D28A9B8DD6A01825728BEDFF0EAAE.png[/IMG]

Table Recognition:

The API can perfectly retrieve the information of tables as well as text from cells, besides, merged cells can also be recognized. It supports recognition of tables with clear and unbroken lines, but not supportive of tables with crooked lines or cells divided by color background. Currently the API supports recognition from printed materials and snapshots of slide meetings, but it is not functioning for the screenshots or photos of excel sheets and any other table editing software.

Here, the image resolution should be higher than 720p (1280×720 px), and the aspect ratio (length-to-width ratio) should be lower than 2:1.

Code Implementation:

As mentioned above sample, we have to Initialize with the VisionBase static class. After that define TableDetector instance and frame. Call the detect method and process the result.

<p style="line-height: 2.0em;">private void setHiAi() {
startTime = System.currentTimeMillis();
/** Define the detector instance and use the Context of this project as a parameter*/
TableDetector detector = new TableDetector(this);
/** Define a frame and initialize it*/
Frame frame = new Frame();

/** Place the bitmap to be checked in the frame*/
frame.setBitmap(bitmap);

/** Check the text and process the result. Note that this line of code must be placed in the working thread rather than the primary thread*/
JSONObject jsonObject = detector.detect(frame, null);
/** Run the convertResult() command to convert the JSON character string into a Java class. You can also parse the JSON character string yourself*/
Table table = detector.convertResult(jsonObject);
System.out.println("Json:"+jsonObject);
extractFromTable(table);
this.txtValue = txtValue;
handler.sendEmptyMessage(Utils.HiAI_RESULT);
}
</p>

Screenshot:

[IMG="https://communityfile-dre.op.hicloud.com/FileServer/getFile/cmtybbs/001/745/226/2640091000001745226.20201030121200.27576941477716278351680519503381:50511111010457:2800:809431033DAB491CFAE3B92EF594BAB6E142B7AB23CCCDC6871A4B0F04CC12C6.png"]https://communityfile-dre.op.hicloud.com/FileServer/getFile/cmtybbs/001/745/226/2640091000001745226.20201030121200.27576941477716278351680519503381:50511111010457:2800:809431033DAB491CFAE3B92EF594BAB6E142B7AB23CCCDC6871A4B0F04CC12C6.png[/IMG]

[IMG="https://communityfile-dre.op.hicloud.com/FileServer/getFile/cmtybbs/001/745/226/2640091000001745226.20201030121212.88205621124823261056866603495903:50511111010457:2800:33AD188F0E78C361DA5368DAF846AA6A46943DFA1DC12AEFAC1985E81E697FDF.png"]https://communityfile-dre.op.hicloud.com/FileServer/getFile/cmtybbs/001/745/226/2640091000001745226.20201030121212.88205621124823261056866603495903:50511111010457:2800:33AD188F0E78C361DA5368DAF846AA6A46943DFA1DC12AEFAC1985E81E697FDF.png[/IMG]

Conclusion:

In this article, we discussed about Document Skew Correction, General Text Recognition and Table Recognition. We can use these in so many real-time scenarios, as follows:

The Document Skew Correction API can be used to easily record works in exhibition regardless of the objective factors such as the angle and the stream of people.

General Text Recognition can provide accurate recognition in various scenarios, even with interferences such as complex light and background situations.

The operational scenarios of Table Recognition are various, so that the third-party apps can generate excel sheets according to the returned results, minimizing manual workload

Reference:

HiAI Guide