Patent Description:
Image recognition systems built in machine such as autonomous vehicles and robots that are essential to full-automation of factories are increasingly used for recognizing objects (recognition targets) needed by the machines. For these image recognition systems, a range of image recognition algorithms for recognizing recognition targets have been developed and studied (PTL1 and NPL1). <NPL>), relates to the proposed development methodology which is addressed by showing the implementation process of an autonomous driving system. In order to describe the implementation process intuitively, core autonomous driving algorithms (localization, perception, planning, vehicle control, and system management) are briefly introduced and applied to the implementation of an autonomous driving system. They are able to examine the advantages of a distributed system architecture and the proposed development process by conducting a case study on the autonomous system implementation. The validity of the proposed methodology is proved through the autonomous car A1 that won the <NUM> Autonomous Vehicle Competition in Korea with all missions completed. <NPL>, relates to a new real-time approach to object detection that exploits the efficiency of cascade classifiers with the accuracy of deep neural networks. Deep networks have been shown to excel at classification tasks, and their ability to operate on raw pixel input without the need to design special features is very appealing. However, deep nets are notoriously slow at inference time. In this paper, they propose an approach that cascades deep nets and fast features, that is both very fast and very accurate. They apply it to the challenging task of pedestrian detection. Their algorithm runs in real-time at <NUM> frames per second. The resulting approach achieves a <NUM>% average miss rate on the Caltech Pedestrian detection benchmark, which is competitive with the very best reported results. It is the first work that achieves very high accuracy while running in real-time.

NPL <NUM>: Basics of Image Pattern Recognition and Machine Vision, Fukuoka System LSI College.

The present disclosure provides an image recognition system that facilitates thermal design of housings and physical layouts.

The image recognition system of the present disclosure is defined in claim <NUM>.

The present disclosure can provide the image recognition system that facilitates thermal design of housings and physical layouts.

Prior to describing exemplary embodiments of the present disclosure, a background to reaching an idea of the present disclosure is described. <FIG> shows an example of vehicle <NUM> equipped with an image recognition system. The image recognition system implements an image recognition algorithm for recognizing a recognition target, such as a pedestrian and other vehicle. Vehicle <NUM> includes on-vehicle camera <NUM> and electronic control unit (ECU) <NUM> that processes an input from on-vehicle camera <NUM>. On-vehicle camera <NUM> and ECU <NUM> send and receive image data and control signal via mutual communication path <NUM>.

<FIG> is a diagram of image recognition system <NUM> that is an example of the image recognition system. On-vehicle camera <NUM> and ECU <NUM> are disposed at positions physically separated from each other. First image processor <NUM> in a housing of on-vehicle camera <NUM> processes an image captured by on-vehicle camera <NUM>. Then, ECU <NUM> receives a result of image processing from on-vehicle camera <NUM> via mutual communication path <NUM>. Then, recognition target detector <NUM> in ECU <NUM> detects a recognition target, and recognition target identifier <NUM> identifies the recognition target with reference to dictionary <NUM> to recognize the recognition target. In image recognition system <NUM>, ECU <NUM> performs image recognition processing.

<FIG> is a diagram of image recognition system 2000A that is another example of the image recognition system. On-vehicle camera 2001A and ECU 2003A are disposed at positions physically separated from each other. First image processor <NUM> in on-vehicle camera 2001A processes an image captured by on-vehicle camera 2001A. Then, recognition target detector <NUM> in on-vehicle camera 2001A detects a recognition target, and recognition target identifier <NUM> in on-vehicle camera 2001A identifies the recognition target with reference to dictionary <NUM> to recognize the recognition target. Then, ECU2003A receives information on the recognition target that is recognized together with image data from on-vehicle camera 2001A via mutual communication path <NUM>. As required, second image processor <NUM> in a housing of ECU2003A performs image processing of the image data. In image recognition system 2000A, on-vehicle camera 2001A performs image recognition processing.

<FIG> is a graph illustrating a relation of a computer operating frequency and a recognition rate of recognition target. As shown in <FIG>, in the image recognition processing, the computer performance generally improves as the computer operating frequency is increased, and thus the recognition rate of recognition target can also be increased. However, a heat amount due to operation also increases as the operating frequency is increased. When the computer operating frequency reaches a certain critical point, heat generated in line with operation cannot be sufficiently released. As a result, a thermal design of the housing of the on-vehicle camera or ECU in which an image recognition means is mounted becomes difficult, and therefore it becomes difficult to install the image recognition means in the on-vehicle camera or ECU.

In other words, integration of image recognition processing that requires a high recognition rate in the camera or ECU will increase an operating frequency required for image recognition processing and a memory capacity required for image recognition processing. As a result, a thermal design of housing and a physical layout become difficult for installing the image recognition means in the housing of the on-vehicle camera or ECU.

Exemplary embodiments of the present disclosure for solving the above disadvantage are described below with reference to drawings.

<FIG> is a block diagram of image recognition system <NUM> according to a first exemplary embodiment of the present disclosure. Image recognition system <NUM> includes first computer <NUM>, second computer <NUM>, and communication path <NUM>. First computer <NUM> is, for example, an on-vehicle camera. Second computer <NUM> is, for example, an ECU.

First computer <NUM> is disposed at a position physically separated from second computer <NUM>. First computer <NUM> and second computer <NUM> are disposed in separate housings.

Communication path <NUM> couples first computer <NUM> and second computer <NUM> in a communicable manner. Communication via communication path <NUM> is typically wireless communication, but wired communication is also applicable.

First computer <NUM> includes recognition target detector <NUM>, first transmitter <NUM>, first receiver <NUM>, first controller <NUM>, and camera <NUM>.

Recognition target detector <NUM> detects a recognition target from an image captured by camera <NUM>. Recognition target detector <NUM> detects the recognition target, using a neural network described later.

First transmitter <NUM> sends data to second computer <NUM>. For example, the sent data includes image data and data of a feature point related to the recognition target included in the image. First receiver <NUM> receives data from second computer <NUM>. For example, the received data includes a detection parameter group included in a recognition parameter group used for image recognition processing. The detection parameter group consists of parameters used for detecting the recognition target from the image.

First controller <NUM> controls operations of first transmitter <NUM>, first receiver <NUM>, and camera <NUM>. Each of recognition target detector <NUM>, first transmitter <NUM>, first receiver <NUM>, and first controller <NUM> may be configured, for example, with hardware that is one or more elements of first computer <NUM>. Alternatively, they may be implemented with programs executed by first computer <NUM>.

Second computer <NUM> includes recognition target identifier <NUM>, second transmitter <NUM>, second receiver <NUM>, second controller <NUM>, and storage unit <NUM>.

Recognition target identifier <NUM> identifies the recognition target from the image captured by camera <NUM>. Recognition target identifier <NUM> identifies the recognition target included in the image based on information including the image data and feature point data. The image data is received via communication path <NUM>, and is data of the image captured by camera <NUM>. The feature point data is detected by recognition target detector <NUM>, which is data of feature point related to the recognition target included in the image. Recognition target identifier <NUM> identifies the recognition target, using a neural network described later.

Second transmitter <NUM> sends data to first computer <NUM>. For example, the sent data includes the detection parameter group. Second receiver <NUM> receives data from first computer <NUM>. For example, the received data includes data of feature point related to the recognition target.

Second controller <NUM> controls operations of recognition target identifier <NUM>, second transmitter <NUM>, second receiver <NUM>, and storage unit <NUM>. Each of recognition target identifier <NUM>, second transmitter <NUM>, second receiver <NUM>, and second controller <NUM> may be configured with, for example, hardware that is one or more elements of second computer <NUM>. Alternatively, they may be implemented with programs executed by second computer <NUM>.

Storage unit <NUM> stores information required for identifying the recognition target as a dictionary. For example, storage unit <NUM> stores dictionary <NUM> as described later with reference to <FIG>.

<FIG> shows an example of communication path <NUM> adopting full-duplex communication system. First transmitter <NUM> of first computer <NUM> is coupled to second receiver <NUM> of second computer <NUM>. First receiver <NUM> of first computer <NUM> is coupled to second transmitter <NUM> of second computer <NUM>. In this case, connections use two systems, and each of first computer <NUM> and second computer <NUM> can simultaneously send and receive data.

<FIG> shows another example of communication path <NUM> adopting the full-duplex communication system. First modem <NUM> is coupled to first transmitter <NUM>, first receiver <NUM>, and second modem <NUM>. Second modem <NUM> is coupled to second transmitter <NUM>, second receiver <NUM>, and first modem <NUM>. First modem <NUM> is, for example, provided inside the housing of first computer <NUM>, and second modem <NUM> is, for example, provided inside the housing of second computer <NUM>.

First modem <NUM> and second modem <NUM> can simultaneously and bidirectionally transfer communication signals on a single bus, typically by frequency division. Accordingly, first computer <NUM>, for example, can perform transmission <NUM> and reception <NUM> simultaneously.

<FIG> shows an example of communication path <NUM> adopting a half-duplex communication system. First switch <NUM> is coupled to second switch <NUM> to switch connection of second switch <NUM> between first transmitter <NUM> and first receiver <NUM>. Second switch <NUM> is connected to first switch <NUM> to switch connection of first switch <NUM> between second transmitter <NUM> and second receiver <NUM>. First switch <NUM> is, for example, provided inside the housing of first computer <NUM>. Second switch <NUM>, for example, is provided inside the housing of second computer <NUM>.

To send data from first computer <NUM>, connection of first switch <NUM> is switched to first transmitter <NUM>, and connection of second switch <NUM> is switched to second receiver <NUM>. To send data from second computer <NUM>, connection of first switch <NUM> is switched to first receiver <NUM>, and connection of second switch <NUM> is switched to second transmitter <NUM>.

As described above, transmission and reception can be switched for communication, using one bus system, in the half-duplex communication system. Although simultaneous communication of sending and receiving is not feasible, a resource required for communication path <NUM> of the half-duplex communication system is smaller than the full-duplex communication system.

First computer <NUM> and second computer <NUM> are coupled by communication path <NUM> that allows bidirectional communication. The communication system of communication path <NUM> is, for example, the full-duplex communication system. In this case, image recognition system <NUM> can easily support real-time processing.

<FIG> is a flow chart illustrating processing of image recognition system <NUM> in the first exemplary embodiment. First, learning data is learned (S1100). Learning of the learning data takes place, for example, using a computer different from both first computer <NUM> and second computer <NUM>. Alternatively, learning data may be learned using second computer <NUM>. For example, the learning data is a feature vector characterizing the recognition target obtained from an image including the recognition target.

Data generated by learning is, in general, called dictionary data. A volume of dictionary data tends to increase according to the number of types of recognition targets (e.g., pedestrians, other vehicles, and obstacles), and a capacity of mounted memory for storing the dictionary data also tends to increase.

Next, second computer <NUM> stores a learning result in storage unit <NUM> as a dictionary (S1200).

Step S1100 and Step S1200 are preliminarily executed during dictionary data creation time <NUM>. This is to execute subsequent Steps S1300 to S1500 for images captured by the camera in a predetermined time, which is identification time <NUM> (preferably real-time execution).

Next, in identification time <NUM>, recognition target detector <NUM> detects the recognition target from the image captured by the camera (S1300), and recognition target identifier <NUM> inputs an identification parameter from the dictionary (S1400) to identify the recognition target (S1500).

<FIG> is a diagram illustrating an example of detection of the recognition target in the image recognition processing. <FIG> is a diagram illustrating an example of identification of the recognition target in the image recognition processing. The image recognition processing is described using an example of character recognition for simplification, with reference to <FIG> and <FIG>, but the recognition target in the image recognition system of the present disclosure is not limited to characters.

For example, handwritten characters have various shapes according to writers, although the same character is written. Therefore, during dictionary data creation time <NUM>, dictionary data creator <NUM> inputs various shapes of the same character (e.g., alphabet "A") to generate feature vectors. These feature vectors are learned to learn features of this character (e.g., alphabet "A"). Then, the dictionary data generated by learning is added to dictionary <NUM> to create dictionary <NUM>. Recognition target identifier <NUM> in image recognition system <NUM> refers to the dictionary data added to dictionary <NUM> for identifying whether or not the character is alphabet "A.

After completing dictionary data creation time <NUM>, recognition target detector <NUM> of first computer <NUM> detects the recognition target in image identification time <NUM>. Recognition target detector <NUM> searches an area to be detected in image <NUM> input from the camera. For example, a target character to be detected (e.g., alphabet "A") is searched while moving search area <NUM> to extract image data <NUM> in the search area including the target character to be detected.

Extracted image data <NUM> is, for example, normalized so as to make the rotated and magnified or reduced character recognizable, and normalized image data <NUM> is generated.

Then, feature vector <NUM> characterizing the target character to be detected (e.g., alphabet "A") included in image data <NUM> is extracted from normalized image data <NUM>, so as to detect the target character to be detected. For example, for extracting features of a two-dimensional image including feature vector <NUM>, methods, such as Histograms of Oriented Gradients (HOG), Scale-Invariant Feature Transform (SIFT), and Speeded-Up Robust Future (SURF) may be used. In HOG, a histogram of gradient directions of luminance of a local area (cell) is generated. In SIFT, the scale transition of the image and the invariant feature amount are extracted. SURF is a high-speed version of SIFT.

Next, recognition target identifier <NUM> identifies alphabet "A" that is target character to be detected <NUM>, typically by pattern matching of extracted feature vector <NUM> using dictionary <NUM>.

<FIG> shows an example of images each including the recognition target of image recognition system <NUM>. As recognition target(s), image 260A includes a pedestrian, image <NUM> includes a vehicle, and image <NUM> includes a pedestrian and obstacle, including a vehicle. Image <NUM> does not include the recognition target.

<FIG> shows an example of a detection area setting using a detection parameter group. To recognize the recognition target, it is necessary to set an appropriate detection area according to the recognition target. Detection area 170a indicates a detection area optimized for detecting a pedestrian in image <NUM>. Detection area 171a indicates a detection area optimized for detecting a vehicle in image <NUM>. Detection area 172a indicates a detection area optimized for detecting an obstacle in image <NUM>. No detection area is set when no recognition target is recognized in image <NUM>.

These detection areas are optimized by optimizing the detection parameter group according to the recognition target. A system for dynamically changing the detection parameter group to an optimum detection parameter group according to the recognition target enables to increase a recognition rate of recognition target in real-time processing. Still more, for example, the control switching between "performing" and "not performing" the image recognition processing for image <NUM> and images <NUM> to <NUM> reduces computation required for image recognition processing. This can suppress power consumption required for image recognition processing. Furthermore, variably change of the detection area required for image recognition processing can make image data not required for image recognition processing to be ignored. This can increase the recognition rate of image recognition processing.

The detection parameter group is, for example, parameters for setting the detection area of image recognition processing. Still more, the detection parameter group may also include parameters for algorithm selection method and extraction algorithm setting, in order to extract the feature amount, such as a feature vector. <FIG> shows an example of the detection parameter group. <FIG> shows an example of feature points of the recognition target.

To detect the recognition target in the image, the detection parameter group includes various parameters as required in addition to a parameter indicating the detection area. The parameter group includes detection parameter group <NUM> optimized for detecting pedestrians, detection parameter group <NUM> optimized for detecting vehicles, detection parameter group <NUM> optimized for detecting obstacles, and detection parameter group <NUM> for not performing detection. Here, "-" means there is no corresponding parameter value. For example, parameter p1 is common to all detection parameter groups <NUM>, <NUM>, <NUM>, and <NUM>, and parameter p6, for example, is only included in detection parameter group <NUM> for detecting obstacles.

These detection parameter groups are updated at intervals of image data transmission (per frame). In other words, the detection parameter groups is updated to detection parameter groups <NUM>, <NUM>, <NUM>, or <NUM> according to the detection target at shooting timing of the camera. According to the updated detection parameter group, first computer <NUM> generates image data and feature point data <NUM>, <NUM>, or <NUM>, based on the detection area, and first computer <NUM> sent them to second computer <NUM>. In this way, first computer <NUM> sends image data to second computer <NUM> for each frame, and the recognition parameter group is changed for each frame.

Parameters for surrounding scenery, time, and so on may also be added to the detection parameter group. This enables first computer <NUM> to optimize detection of the recognition target by referring to the detection parameter group, even though the surrounding scenery or time changes. By sending optimum detection parameter group for detecting the recognition target to first computer <NUM> before detecting the recognition target, the recognition rate can be increased.

<FIG> is a flow chart of communication between first computer <NUM> and second computer <NUM>. The communication flow shown in <FIG> is for communication adopting the full-duplex communication system. Second controller <NUM> generates, for example, control signals <NUM> and <NUM> including the detection parameter group, and transfers them to second transmitter <NUM>. Second transmitter <NUM> sends transferred control signals <NUM> and <NUM> to first receiver <NUM> of first computer <NUM>. Here, the time required for transferring control signals <NUM> and <NUM> within second computer <NUM> is negligibly small, compared to the time required for sending from second transmitter <NUM> to first receiver <NUM>.

First receiver <NUM> transfers received control signals <NUM> and <NUM> to first controller <NUM>. Here also, the time required for transfer within first computer <NUM> is negligibly small, compared to the time required for sending from second transmitter <NUM> to first receiver <NUM>. First controller <NUM> controls camera <NUM>, based on control signals <NUM> and <NUM>, to generate image data <NUM> and <NUM>. Recognition target detector <NUM> extracts feature points from image data <NUM> and <NUM> to generate feature point data <NUM>.

Next, first transmitter <NUM> sends image data <NUM> and <NUM> and feature point data <NUM> to second receiver <NUM> of second computer <NUM>. Second receiver <NUM> transfers received image data <NUM> and <NUM> to recognition target identifier <NUM>. Here also, the time required for transfer is negligibly small, compared to the time required for sending from second transmitter <NUM> to first receiver <NUM>. Recognition target identifier <NUM> identifies the recognition target based on image data <NUM> and <NUM> and feature point data <NUM>.

For transmission from second transmitter <NUM> to first receiver <NUM> and from first transmitter <NUM> to second receiver <NUM>, a certain time is required. However, in the full-duplex communication system, transmission of control signal <NUM> and transmission of image data <NUM> and feature point data <NUM> take place in parallel. In the same way, transmission of control signal <NUM> and transmission of image data <NUM> and feature point data <NUM> take place in parallel.

Still more, as shown in <FIG>, reception of control signal <NUM> by first receiver <NUM> and detection by recognition target detector <NUM> (generation of image data <NUM> and feature point data <NUM>) take place in parallel processing. Furthermore, transmission of control signal <NUM> by second transmitter <NUM> and identification of the recognition target by recognition target identifier <NUM> take place in parallel processing. By communication and parallel processing using the full-duplex communication system, latency of communication between first computer <NUM> and second computer <NUM> can be concealed, even when first computer <NUM> and second computer <NUM> are disposed at positions physically separated from each other. Consequently, throughput of the computers can be efficiently demonstrated.

<FIG> illustrates image recognition processing in image recognition system <NUM> according to the first exemplary embodiment. In the image recognition processing, as shown in <FIG>, the detection area in image data <NUM> is searched, pattern matching is performed between dictionary data and feature point data <NUM> of the recognition target detected in the detection area, and a pattern-matching result is output as recognition result <NUM> of the recognition target. In this image recognition processing, image data varies, typically due to individual difference between cameras and correlation among camera attachment position, background image, location, and so on, when feature point data is extracted. This makes processing complicated and increases computational effort. Also on computing recognition result <NUM> of the recognition target from feature point data <NUM>, similar variations in image data occur, typically due to individual difference between cameras and correlation among camera attachment position, background image, location, and so on. This also makes processing complicated and increases computational effort.

In other words, to integrally perform image recognition processing by a processor, such as ECU, using image data obtained by single or multiple cameras, any difference caused by variations in image data, typically due to individual difference between cameras used and camera attachment position, needs to be eliminated. Accordingly, computation effort required for recognition and also resulting heat amount increase.

In the present exemplary embodiment, as shown in <FIG>, feature point data <NUM> is extracted in first computer <NUM>, and computation of recognition result <NUM> is performed in second computer <NUM>. The image recognition processing is thus distributed between first computer <NUM> and second computer <NUM>. Therefore, even though there are variations among image data as described above, heat amount and outer dimensions of housing of each of first computer <NUM> and second computer <NUM> can be reduced, compared to the use of an integral computer. Accordingly, even when the operating frequency or memory capacity required for recognition is increased in order to increase the recognition rate of recognition target, the present disclosure facilitates thermal design and physical layout on installing image recognition system <NUM>. This effect is particularly remarkable when multiple cameras are used.

In the present exemplary embodiment, processing needed for image recognition is performed by parallel processing using first computer <NUM> and second computer <NUM> that are disposed at positions physically separated from each other. This facilitates thermal design and physical layout for installing image recognition system <NUM>, although computation effort needed for image recognition and data transmission are increased. Still more, thermal design and physical layout for installing image recognition system <NUM> can also be further facilitated, even when the capacity of the memory for storing images required for image recognition is increased or dictionary data that increases in line with the number of types of recognition targets to be recognized and the memory size for storing the dictionary data are increased. As a result, the recognition rate of recognition target of image recognition system <NUM> can be further easily increased.

Still more, when an algorithm called Structure from Motion (SfM) is used for detection or identification of the recognition target, the recognition target is extracted based on a difference in multiple pieces of image data. Accordingly, the capacity of a mounted memory required for executing a recognition algorithm tends to increase depending on a detection rate of recognition algorithm adopted. Also in such a case, thermal design and physical layout for installing image recognition system <NUM> can be further facilitated.

<FIG> is a graph showing the relation between the recognition rate of recognition target and mounted memory size (capacity). In general, in the image recognition processing, multiple images required for image recognition are stored in the mounted memory. Therefore, as the number of types of recognition targets to be recognized or the number of recognition targets increases, a required mounted memory size tends to increase. In addition, to increase the recognition rate of recognition target, a required mounted memory size tends to increase. When the required mounted memory size becomes equal to or more than a certain size, a physical area and outer dimensions to be needed enlarge and the memory may become impossible to be installed in the housing.

Also in this case, image recognition system <NUM> can reduce a physical area and outer dimensions of each of first computer <NUM> and second computer <NUM>, compared to the use of an integral computer, and thus the computers can be easily installed in the housings.

Furthermore, in image recognition system <NUM>, the recognition target can be dynamically changed according to various occasions, and thus real-time recognition rate of recognition target can be increased.

Further example For the image recognition algorithm in image recognition system <NUM> described in the first exemplary embodiment, deep learning can also be adopted. To increase a recognition rate on constructing the image recognition system, an uniquely-tuned recognition parameter group is provided for each of the detection area, the feature extraction, the dictionary data creation, the pattern matching, and so on. A designer who constructs the image recognition system manually prepares each recognition parameter group, assuming target images to be recognized and operating conditions. The recognition rate of recognition target also differs by a camera to be used and setting of recognition parameter group according to occasion.

Still more, for real-time recognition of different recognition targets in the image recognition system used in automated driving vehicles and robots, it is important to increase the recognition rate of real-time recognition processing. There are various indexes for recognition rate, but for image recognition processing of moving recognition targets, such as moving objects, it is particularly important to improve recognition rate of real-time recognition processing.

For this reason, a method called deep learning has been increasingly drawing attention as a method that reduces manual preparation or achieves full-automated preparation of recognition parameter groups in construction of such image recognition system.

To increase the recognition rate in real-time recognition processing, a volume of dictionary data needs to be increased, and thus a volume of learning data used for computation of recognition parameter groups also increases. Therefore, a large volume of data, such as learning data, is often stored in a server as big data. It is difficult to manually optimize recognition parameter groups for image data recognition algorithm, using this large volume of data. Instead of having a designer constructing image recognition system set recognition parameter groups as intended, recognition algorithm and recognition rules are automatically generated, using stored large volume of data. Using this auto-generated recognition algorithm and rules, meaning of input image is automatically predicted.

<FIG> shows an example of neural network used for deep learning. In contrast to a conventional neural network having, for example, three-layer structure, deep learning has a neural network of relatively deep layered structure. Neurons <NUM> are coupled between adjacent layers by synapse <NUM>.

An example of mathematical model of single neuron is expressed with Formula (<NUM>) below.

Where, "y" represents an output signal value of certain neuron <NUM>, "fk" represents a function, such as the sigmoid function, "n" represents the number of neurons in lower layers under certain neuron <NUM>, "xi" represents an output signal value of ith neuron <NUM> in the lower layers, "wi" represents a synapse weight of synapse <NUM> connected from ith neuron <NUM> in the lower layer to certain neuron <NUM>, and "θ" is certain threshold.

It is apparent from above Formula (<NUM>) that neuron <NUM> fires when a value of product-sum calculation of neuron <NUM> in the lower layer and synapse weight of synapse <NUM> exceeds threshold θ, and its signal propagates through neural networks <NUM> and <NUM>. For example, neural network <NUM> is used for the image recognition processing, and neural network <NUM> is used for action control. In this case, a value of synapse weight "wi" serves as a recognition parameter group for image recognition.

First, image data stored typically in an external server is used for learning in hierarchical neural networks <NUM> and <NUM> by deep learning. Then, based on a neural network structure formed by learning and synapse weight, a neural network structure is obtained. Recognition data <NUM> of image recognition is output from neural network <NUM> for image recognition processing.

In the deep learning, the recognition rate of recognition target increases by optimizing the synapse weight of neural network specified for the recognition target to be recognized. Therefore, based on change data <NUM> of synapse weight, the recognition parameter group optimized for each recognition target to be recognized is dynamically changed. This can increase a real-time recognition rate of recognition target.

In the recognition parameter group based on deep learning, function "fk" in Formula (<NUM>) may be included, or threshold "θ" may be included.

In a general neuron model shown by Formula (<NUM>), a network structure may be dynamically changed by setting at least one of the synapse weights to <NUM>(zero) to change the network structure of neural network.

As described above, first computer <NUM> detects the recognition target by employing the neural network. In this case, the detection parameter group is at least one of the synapse weight, function, and threshold of neuron configuring the neural network.

Further example <FIG> is image recognition system <NUM> according to a third exemplary embodiment of the present disclosure. Image recognition system <NUM> includes first computer <NUM>, second computer <NUM>, and camera <NUM>. Second computer <NUM> in the third exemplary embodiment has the same configuration as second computer <NUM> in the first exemplary embodiment, and thus its description is omitted here.

First computer <NUM> in the first exemplary embodiment includes camera <NUM> inside. Conversely, first computer <NUM> in the third exemplary embodiment is connected to separate camera <NUM>. First computer <NUM> in the third exemplary embodiment differs from first computer <NUM> in the first exemplary embodiment at this point. By providing first computer <NUM> and camera <NUM> separately, image recognition system <NUM> in the third exemplary embodiment can further prevent concentration of heat generated due to processing, compared to that of image recognition system <NUM> in the first exemplary embodiment. In this way, camera <NUM> for obtaining image data may be disposed physically separated from first computer <NUM> and connected to first computer <NUM>.

Further example <FIG> is image recognition system 500A according to a fourth exemplary embodiment of the present disclosure. Image recognition system 500A includes first computer <NUM>, second computer <NUM>, and server <NUM>. First computer <NUM> in the fourth exemplary embodiment has the same configuration as first computer <NUM> in the first exemplary embodiment, and thus its description is omitted here. Second computer <NUM> obtains part or all of recognition parameter groups from server <NUM> connected to second computer <NUM>, instead of storing the recognition parameter groups in a memory of recognition target identifier <NUM>. As required, second computer <NUM> sends a result of image recognition by second computer <NUM> to server <NUM>.

To increase the recognition rate for various recognition targets, (e.g., pedestrians, other vehicles, and bicycles), a dictionary needs to contain a large volume of dictionary data. By storing the dictionary in server <NUM> and obtaining dictionary data from the dictionary stored in server <NUM> by second computer <NUM> as required, an increase in a data memory size of recognition target identifier <NUM> in second computer <NUM> performing image recognition and an increase in a memory capacity of storage unit <NUM> can be suppressed. This can suppress a heat generated from data memory, and also the outer dimensions of second computer <NUM> can be small. Still more, when the recognition parameter group changes rapidly, image recognition system 500Acan obtain the latest recognition parameter group from server <NUM> and use it.

Furthermore, to perform image recognition in a moving vehicle, for example, image data from the vehicle of various places to which the vehicle travels at different times are needed. This generally requires a large volume of image data for creating a dictionary. Also in this case, image data from the vehicle can be stored in server <NUM> and create the dictionary in server <NUM>.

Further example <FIG> shows image recognition system <NUM> according to a fifth exemplary embodiment of the present disclosure. Image recognition system <NUM> includes first camera <NUM>, second camera <NUM>, third camera <NUM>, and fourth camera <NUM> as first computers, ECU <NUM> as a second computer, and ranging sensor <NUM>. First camera <NUM>, second camera <NUM>, third camera <NUM>, fourth camera <NUM>, and ranging sensor <NUM> are connected to ECU <NUM>.

For example, when a vehicle moves forward, fourth camera <NUM> detects an obstacle ahead, second camera <NUM> and third camera <NUM> are turned off, and first camera <NUM> detects a following vehicle. When the vehicle turns to the right, fourth camera <NUM> detects an obstacle ahead, third camera <NUM> detects a pedestrian at the right to avoid an accident due to turning, second camera <NUM> detects a vehicle at the left, and first camera <NUM> detects a following vehicle.

For example, ranging sensor <NUM> is connected to ECU <NUM>. Image data of fourth camera <NUM> and ranging data of ranging sensor <NUM> may be synthesized to prepare three-dimensional image to be used for detecting an obstacle ahead. Ranging sensor <NUM> is capable of measuring a distance to a recognition target, and is, for example, a milliwave sensor or a sonar sensor. By constructing the image recognition system using multiple cameras <NUM>, <NUM>, <NUM>, and <NUM>, and ranging sensor <NUM>, the recognition rate of recognition target can be further increased.

Further examples First computer <NUM> in the first exemplary embodiment performs detection, but first computer <NUM> may additionally perform part of identification, instead of second computer <NUM>.

First computer <NUM> in the first exemplary embodiment includes camera <NUM>. In addition to camera <NUM>, for example, LIDAR (Light Detection and Ranging or Laser Imaging Detection and Ranging) may be used to combine image data from camera <NUM> and image data from LIDAR and to prepare three-dimensional image data to be used for detecting and/or identifying a recognition target.

Claim 1:
An image recognition system (<NUM>) comprising:
a first computer (<NUM>, <NUM>) which, in operation, detects a recognition target and feature point data (<NUM>, <NUM>, <NUM>) from a detection area (170a, 171a, 172a) of image data (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>), using a detection parameter group (<NUM>, <NUM>, <NUM>, <NUM>) for setting the detection area (170a, 171a, 172a) and for detecting the feature point data (<NUM>, <NUM>, <NUM>), the recognition target comprising at least one of a pedestrian, a vehicle and an obstacle;
a second computer (<NUM>, <NUM>) which, in operation, identifies the recognition target by performing pattern matching between dictionary data and the feature point data (<NUM>, <NUM>, <NUM>) detected by the first computer; and
a communication path (<NUM>) between the first computer (<NUM>, <NUM>) and the second computer (<NUM>, <NUM>) wherein
the first computer (<NUM>, <NUM>) and the second computer (<NUM>, <NUM>) are disposed in separate housings at positions physically separated from each other,
the first computer uses a neural network,
the detection parameter group (<NUM>, <NUM>, <NUM>, <NUM>) consists of parameters (p1, p2, p3, p4, p5, p6) used for detecting the recognition target from the image data (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>), wherein the detection parameter group (<NUM>, <NUM>, <NUM>, <NUM>) is included in a recognition parameter group, and is dynamically changed according to the recognition target, and used for image recognition processing of the image data (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>), wherein the recognition parameter group is at least a part of a parameter group that characterizes the recognition target to be recognized,
the detection parameter group (<NUM>, <NUM>, <NUM>, <NUM>) is at least one of a synapse weight, a function, and a threshold of a neuron configuring the neural network, wherein the synapse weight is one of a plurality of synapse weights, and a configuration of the neural network is changed by setting zero to at least one of the plurality of synapse weights; and
the second computer (<NUM>, <NUM>) sends the detection parameter group (<NUM>, <NUM>, <NUM>, <NUM>) to the first computer (<NUM>, <NUM>) via the communication path (<NUM>) when the first computer (<NUM>, <NUM>) sends the image data (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>) to the second computer (<NUM>, <NUM>).