Image recognition device and method for recognizing images

A method for detecting and recognizing objects in images includes obtaining a video stream and pre-processing the video stream to obtain an image queue arranged in a frame playing order and storing the image queue into a storage device. An image frame of the image queue from the storage device is read and at least one object in the image frame is detected and recognized. An image recognition device is also provided.

FIELD

The subject matter herein generally relates to imaging recognition.

BACKGROUND

A video frame rate of a 4K (or 8K) video playing device can be 30 FPS, or higher. When one or more object detections are required for each video frame, the image recognition workload is large and the recognition speed is required to be fast. Current image recognition systems are not fast enough for real-time object detection.

Thus, there is a room for improvement.

DETAILED DESCRIPTION

It will be appreciated that for simplicity and clarity of illustration, where appropriate, reference numerals have been repeated among the different figures to indicate corresponding or analogous elements. In addition, numerous specific details are set forth in order to provide a thorough understanding of the embodiments described herein. However, it will be understood by those of ordinary skill in the art that the embodiments described herein can be practiced without these specific details. In other instances, methods, procedures, and components have not been described in detail so as not to obscure the related relevant feature being described. Also, the description is not to be considered as limiting the scope of the embodiments described herein. The drawings are not necessarily to scale and the proportions of certain parts may be exaggerated to better illustrate details and features of the present disclosure. It should be noted that references to “an” or “one” embodiment in this disclosure are not necessarily to the same embodiment, and such references mean “at least one”.

The connection can be such that the objects are permanently connected or releasably connected. The term “comprising,” when utilized, means “including, but not necessarily limited to”; it specifically indicates open-ended inclusion or membership in the so-described combination, group, series, and the like.

FIG. 1illustrates an image recognition device100for recognizing at least one object in an image in accordance with an embodiment. The image recognition device100can comprise a first processing module10, a storage device20, a second processing module30, and a communication bus40. The first processing module10, the storage device20, and the second processing module30can communicate with each other through the communication bus40.

In one embodiment, the storage device20can be inside the image recognition device100, or can be a separate external memory card, such as an SM card (Smart Media Card), an SD card (Secure Digital Card), or the like. The storage device10can include various types of non-transitory computer-readable storage mediums. For example, the storage device10can be an internal storage system, such as a flash memory, a random access memory (RAM) for temporary storage of information, and/or a read-only memory (ROM) for permanent storage of information. The storage device10can also be an external storage system, such as a hard disk, a storage card, or a data storage medium. The communication bus40can comprise data buses, power buses, and control buses.

The first processing module10can obtain a video stream and pre-process the video stream to obtain an image queue arranged in a frame playing order.

In one embodiment, the first processing module10can obtain the video stream from a video recording device, a video playing device, or a data storage device. The data storage device can be the storage device10, thus, the first processing module10can obtain the video stream from the storage device20. The pre-processing can comprise decoding the video stream and segmenting the video stream into a plurality of image frames.

In one embodiment, the processing module10can pre-process the video stream to obtain an image queue arranged in a frame playing order. The starting image frame of the image queue can be the first frame image of the video stream, and the end image frame of the image queue can be the last frame image of the video stream.

In one embodiment, the processing module10further stores the image queue in the storage device10. The processing module10can comprise one or more processors, the processor can be a central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a complex programmable logic device (CPLD). When the processing module10comprise multiple processors, the processing module10can pre-process the video stream using multiple threads.

The second processor30is configured to read an image frame of the image queue from the storage device20and recognize the image frame to detect at least one object in the image frame.

In one embodiment, the second processor30can comprise at least one image recognition model. The second processor30can recognize the image frame to detect the at least one object using the at least one image recognition model. The image recognition model can be a recognition model trained by a machine learning algorithm, such as a depth residual network algorithm, or a convolutional neural network algorithm, etc.

In one embodiment, the object recognized by the second processor30can be defined according to actual use requirements of the image recognition device100. For example, the object can be a person, a container, or a handbag. When the second processor30need to identify multiple types of objects, each type of object may correspond to one image recognition model.

For example, the second processor30comprises a first image recognition model and a second image recognition model. The first image recognition model is configured for recognizing persons in the image frame, and the second image recognition model is configured for recognizing containers in the image frame. Thus, the second processing module30can realize detection of persons and of containers in the image frame.

In one embodiment, the image frame read by the second processing module30can be a frame of the image queue that is not being recognized.

In one embodiment, the second processing module30can repeatedly read the image frame one by one in sequence according to a frame order of the image queue. That is, the image frames ranked in the top of the image queue are preferentially read and recognized by the second processing module30.

In one embodiment, when the second processing module30reads the image frame of the image queue from the storage device20, the second processing module30further adjusts image parameters of the image frame and recognizes the adjusted image frame to detect the at least one object. The image parameters can comprise a pixel parameter and/or a brightness parameter. For example, the second processing module30can reduce pixels of the image frame to improve a recognition speed of the image frame.

In one embodiment, the second processing module30can recognize the image frame to obtain information as to category of the at least one object and save the category information as a label of the at least one object into the image frame. For example, when the second processing module30recognizes that an object is a person, the category information of the object can be “person”. When the second processing module30recognizes that an object is a handbag, the category information of the object can be “container”.

Referring toFIG. 2, the second processing module30can be a processor clustering, and the second processing module30supports hardware extension to improve data processing capability. The second processing module30can comprise a plurality of threads P#1, P#2, . . . , P#N (P#1˜P#N) and a plurality of processing units PU#1, PU#2, . . . , PU#N (PU#1˜PU#N). N can be a natural number that is greater than1. Each of the plurality of threads P#1˜P#N corresponds to one of the plurality of processing units PU#1˜PU#N. For example, the thread P#1reads an image frame of the image queue from the storage device20and transmits the image frame to the processing unit PU#1to detect the at least one object.

In one embodiment, each of the plurality of processing units PU#1˜PU#N can comprise at least one image recognition model. The second processing module30has a good recognition speed by the plurality of threads P#1˜P#N and the plurality of processor units PU#1˜PU#N, to realize real-time object detection with regard to ultra-high-definition (UHD) image frames.

In one embodiment, when the thread P#1reads a UHD image frame from the storage device20, the thread P#1transmits the UHD image frame to the processing unit PU#1to detect the at least one object. In order to avoid having to read large amount of data, leading to low image recognition speed, the thread P#1reads the next UHD image frame after the processing unit PU#1has recognized the current UHD image frame.

In one embodiment, when the storage device20stores unrecognized image frames, the plurality of processing units PU#1˜PU#N can read the image frame in parallel. For example, the second processing module30comprises eight threads P#1˜P#8and eight processing units PU#1˜PU#8. The thread P#1reads a first image frame of the image queue and transmits same to the processing units PU#1, the thread P#2reads a second image frame of the image queue and transmits same to the processing units PU#2, and the thread P#8reads an eighth image frame of the image queue and transmits same to the processing units PU#8. When the processing unit PU#1has recognized the first image frame of the image queue, the thread P#1reads a ninth image frame of the image queue and transmits same to the processing unit PU#1.

In one embodiment, each of the processor units PU#1˜PU#N can be a CPU, a microprocessor, an ASIC, a FPGA, or a CPLD.

FIG. 3illustrates one exemplary embodiment of an image recognition method. The flowchart presents an exemplary embodiment of the method. The exemplary method is provided by way of example, as there are a variety of ways to carry out the method. The method described below can be carried out using the configurations illustrated inFIG. 1, for example, and various elements of these figures are referenced in explaining the example method. Each block shown inFIG. 3may represent one or more processes, methods, or subroutines, carried out in the example method. Furthermore, the illustrated order of blocks is illustrative only and the order of the blocks can change. Additional blocks can be added or fewer blocks may be utilized, without departing from this disclosure. The example method can begin at block300.

In block300, obtaining a video stream and pre-processing the video stream to obtain an image queue arranged in a frame playing order by the first processing module10.

In block302, storing the image queue into the storage device20by the first processing module10.

In block304, reading an image frame of the image queue from the storage device20and recognizing the image frame to detect at least one object in the image frame by the second processing module30.