Patent Description:
Autonomous driving of a vehicle enables various driving operations to be automatically performed. For example, an autonomous host vehicle independently travels on a road without a driver operating the vehicle through a steering wheel, an accelerator pedal, or a brake. An object recognition for autonomous driving is performed based on image information analyzed in a vehicle.

In one general aspect, there is provided an object recognition method as defined by claim <NUM>.

In another general aspect, there is provided a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the method.

In another general aspect, there is provided an object recognition apparatus, as defined by claim <NUM>.

The following structural or functional descriptions of examples disclosed in the present disclosure are merely intended for the purpose of describing the examples and the examples may be implemented in various forms. The examples are not meant to be limited, but it is intended that various modifications, equivalents, and alternatives are also covered within the scope of the claims.

Although terms of "first" or "second" are used to explain various components, the components are not limited to the terms. These terms should be used only to distinguish one component from another component. For example, a "first" component may be referred to as a "second" component, or similarly, and the "second" component may be referred to as the "first" component within the scope of the right according to the concept of the present disclosure.

As used herein, the singular forms are intended to include the plural forms as well, unless the context clearly indicates otherwise.

In an example, the object recognition method and apparatuses is implemented in various types of products, such as, for example, an intelligent agent, a mobile phone, a cellular phone, a smart phone, a wearable smart device (such as, a ring, a watch, a pair of glasses, glasses-type device, a bracelet, an ankle bracket, a belt, a necklace, an earring, a headband, a helmet, a device embedded in the cloths, or an eye glass display (EGD)), a server, personal computers (PC), laptop computers, tablet computers, a laptop, a notebook, a subnotebook, a netbook, an ultra-mobile PC (UMPC), a tablet personal computer (tablet), a phablet, a mobile internet device (MID), a personal digital assistant (PDA), an enterprise digital assistant (EDA), a digital camera, a digital video camera, a portable game console, an MP3 player, a portable/personal multimedia player (PMP), a handheld e-book, an ultra mobile personal computer (UMPC), a portable lab-top PC, a global positioning system (GPS) navigation, a personal navigation device, portable navigation device (PND), a handheld game console, an e-book, televisions (TVs), a high definition television (HDTV), a smart appliance, a smart home appliances, an intelligent vehicles, kiosks, a biometrics-based door lock, a security device, a financial service device, communication systems, image processing systems, graphics processing systems, various Internet of Things (IoT) devices that are controlled through a network, a smart vehicle, other consumer electronics/information technology(CE/IT) device, or any other device capable of wireless communication or network communication consistent with that disclosed herein.

In an example, the object recognition apparatus described herein may be incorporated in a vehicle. The vehicle described herein refers to any mode of transportation, delivery, or communication such as, for example, an automobile, a truck, a tractor, a scooter, a motorcycle, a cycle, an amphibious vehicle, a snowmobile, a boat, a public transit vehicle, a bus, a monorail, a train, a tram, an autonomous or automated driving vehicle, an intelligent vehicle, a self-driving vehicle, an aircraft, an unmanned aerial vehicle, a drone, or a mobile device. Also, examples may be used to provide information for autonomous driving of an intelligent vehicle by recognizing an object and control an autonomous vehicle. In an example, the object recognition apparatus is applicable to a robot requiring a positioning operation.

The apparatus and methods described herein may be used to recognize an object in a navigation system of a smart vehicle, to generate location information to assist an autonomous or automated driving vehicle in steering, for in-vehicle driving assistance for fully autonomous or automated driving, and thus, enable safer and more comfortable driving.

<FIG> illustrates an example of a network architecture of an object recognition apparatus <NUM>. Referring to <FIG>, the object recognition apparatus <NUM> includes a faster region-based convolutional neural network (R-CNN) <NUM> and a processor <NUM>.

In an example, the faster R-CNN <NUM> receives all regions of an input image <NUM> and processes object candidate regions corresponding to the input image <NUM> at once. The faster R-CNN <NUM> extracts features from all the regions of the input image <NUM> at once through max-pooling in a CNN <NUM> including a plurality of convolutional (conv) layers, and generates a feature map <NUM>. In an example, a feature map <NUM>, or a plurality of feature maps <NUM> may be provided. The feature map <NUM> is, for example, a Conv5 feature map.

The faster R-CNN <NUM> obtains candidate regions with a high probability that an object of interest exists, i.e., obtains proposals <NUM> from the feature map <NUM> using a region proposal network (RPN) <NUM>. In the following description, the proposals <NUM> are referred to as "object candidate regions. " A configuration of the RPN <NUM> will be described in detail below with reference to <FIG>.

In an example, the faster R-CNN <NUM> provides the object candidate regions obtained using the RPN <NUM> to an ROI pooling layer <NUM>. In an example, the faster R-CNN <NUM> extracts fixed-length feature vectors from feature maps <NUM> through the ROI pooling layer <NUM>. In an example, the extracted fixed-length feature vectors are applied to a fully-connected (FC) layer (not shown). In an example, the faster R-CNN <NUM> includes a classifier <NUM> configured to estimate an object class and a background, and a bounding box regressor (not shown) configured to output a position of each object class. The classifier <NUM> is, for example, a softmax classifier. In an example, the ROI pooling layer <NUM> and the classifier <NUM> correspond to a detection network configured to recognize an object. The classifier <NUM> and the bounding box regressor are connected to a rear end of the FC layer.

The faster R-CNN <NUM> performs a convolution operation with respect to all the regions of the input image <NUM> only once by the CNN <NUM>, and shares a result of the convolution operation in the faster R-CNN <NUM>. The ROI pooling layer <NUM> adjusts a size so that various object candidate regions are input to the FC layer.

Generally, an inference time used to infer an object in the faster R-CNN <NUM> is approximately expressed using an equation "Inference Time ≈ <NUM> × ConvTime + Num of Proposals × fcTime. " In the equation, ConvTime denotes a time used to perform a convolution operation in the CNN <NUM>, and fcTime denotes a time used in an FC layer. An inference time of the faster R-CNN <NUM> is proportional to a number of proposals, i.e., a number of object candidate regions.

The object recognition apparatus <NUM> reduces a number of object candidate regions, which has a significant influence on the inference time in the faster R-CNN <NUM> in proportion to an area of a region of interest (ROI). Thus, enhancing a speed of the inference time.

The object recognition apparatus <NUM> improves the object recognition speed for example, by quickly extracting a road region in which a vehicle travels from the input image <NUM>, and performing an object recognition with respect to the road region. The object recognition apparatus <NUM> sets the extracted road region as an ROI, resets a number of object candidate regions used in the faster R-CNN <NUM> to be suitable for a size of the ROI, and performs the object recognition, to effectively enhance the object recognition speed. The road region set as the ROI is extracted using a scene segmentation algorithm performed by the processor <NUM>. The processor <NUM> determines a number of object candidate regions based on the size of the ROI and provides the number of object candidate regions to the RPN <NUM>. In an example, the above operation of the processor <NUM> and an operation of generating the feature map <NUM> in the faster R-CNN <NUM> are performed in parallel or sequentially.

In an example, when the operation of the faster R-CNN <NUM> and the operation of the processor <NUM> are performed in parallel, an additional amount of time to extract the ROI and determine the number of object candidate regions is not calculated.

In another example, the processor <NUM> extracts an ROI at a relatively high speed within <NUM> milliseconds (ms) using various ROI extraction schemes that are based on a computer vision algorithm, and operates sequentially with the faster R-CNN <NUM>.

Hereinafter, an example in which the operation of the faster R-CNN <NUM> and the operation of the processor <NUM> are performed in parallel will be described below with reference to <FIG> and <FIG>, and an example in which the operation of the faster R-CNN <NUM> and the operation of the processor <NUM> are sequentially performed will be described below with reference to <FIG> and <FIG>.

<FIG> illustrates an example of an operation of the RPN <NUM>. Referring to <FIG>, in the RPN <NUM>, a size of an input image is not limited, and an output is a set of rectangular objects, each with an "objectness score," for each object candidate region. The "objectness score" corresponds to a probability (for example, <NUM> or <NUM>) that an object of interest exists in a corresponding region. A model of the RPN <NUM> is, for example, in a form of a fully convolutional network.

The RPN <NUM> receives, as an input, a feature map <NUM> of a CNN including convolutional layers, performs a convolution operation by an intermediate layer <NUM> using a sliding window <NUM> with a size of "n × n," and generates a <NUM>-dimensional (or <NUM>-dimensional) feature vector. The <NUM>-dimensional feature vector is applied to each of a classification(cls) layer <NUM> and a regression(reg) layer <NUM>. The cls layer <NUM> indicates whether <NUM>-dimensional feature vectors represent an object through a box classification. The reg layer <NUM> generates coordinates of object candidate regions corresponding to <NUM>-dimensional feature vectors.

For example, the cls layer <NUM> obtains an objectness score indicating whether each of "k" object candidate regions corresponds to an object. Accordingly, an output value of the cls layer <NUM> is "<NUM> scores. " Also, the reg layer <NUM> outputs four coordinate values (X, Y, W, H) of each object candidate region. Accordingly, an output value of the reg layer <NUM> is "<NUM> coordinates.

Depending on examples, a total of "k" object candidate regions are recommended for each sliding window <NUM>. The total of "k" object candidate regions recommended for each sliding window <NUM> correspond to combinations (for example, "k" anchor boxes <NUM>) in which a scale and an aspect ratio of the sliding window <NUM> varies based on a center of the sliding window <NUM>.

<FIG> illustrates an example of an object recognition method. The operations in <FIG> may be performed in the sequence and manner as shown, although the order of some operations may be changed or some of the operations omitted without departing from the scope of the illustrative examples described. Many of the operations shown in <FIG> may be performed in parallel or concurrently. One or more blocks of <FIG>, and combinations of the blocks, can be implemented by special purpose hardware-based computer that perform the specified functions, or combinations of special purpose hardware and computer instructions. In addition to the description of <FIG> below, the descriptions of <FIG> are also applicable to <FIG>, and are incorporated herein by reference. Thus, the above description may not be repeated here.

Referring to <FIG>, in operation <NUM>, an object recognition apparatus extracts a feature vector from an input image and generates a feature map in a neural network. The neural network includes a plurality of layers. The neural network includes an R-CNN including an RPN and a detection network.

In operation <NUM>, in parallel with the generating of the feature map in operation <NUM>, the object recognition apparatus extracts, using a processor, an ROI and, determines a number of object candidate regions. Operations performed in parallel with the generating of the feature map will be further described below with reference to <FIG>. In operation <NUM>, the object recognition apparatus extracts an ROI corresponding to at least one object of interest from the input image. The object of interest includes objects such as, for example, a road, a vehicle, a human, an animal, a plant, or a building. Also, the ROI is, for example, a region corresponding to a road, a vehicle, a human, an animal, a plant and a building. In an example, the object recognition apparatus uses a training-based scene segmentation algorithm and an image processing algorithm to extract an ROI.

In operation <NUM>, the object recognition apparatus determines, based on a size of the ROI, a number of object candidate regions that are used to detect an object of interest. The object recognition apparatus determines the number of object candidate regions based on the size of the ROI and a size of the input image. The object recognition apparatus calculates a ratio of the size of the ROI (denoted by Road Area) to the size of the input image (denoted by Input Image Area) and determines the number of object candidate regions based on the calculated ratio, as shown in Equation <NUM> below.

In operation <NUM>, the object recognition apparatus recognizes the object of interest based on the number of object candidate regions determined in operation <NUM>. The object recognition apparatus determines positions of the object candidate regions on the feature map output from the neural network. The object recognition apparatus determines a position, i.e., coordinates of an object candidate region using the above-described RPN. The object recognition apparatus recognizes the object of interest from the ROI based on the position of the object candidate region.

When the object of interest is recognized, only the extracted ROI (for example, a road region) is used, instead of all regions of the input image. Thus, a number of object candidate regions are reduced, and an amount of time for object recognition is greatly reduced.

In an example, the object recognition apparatus determines a control parameter used to control a speed of a vehicle and a traveling direction of the vehicle based on an object recognition result of operation <NUM>, and controls the running of the vehicle using the control parameter.

<FIG> illustrates an example of a process of performing operations of an object recognition method in parallel. The operations in <FIG> may be performed in the sequence and manner as shown, although the order of some operations may be changed or some of the operations omitted without departing from the scope of the illustrative examples described. Many of the operations shown in <FIG> may be performed in parallel or concurrently. One or more blocks of <FIG>, and combinations of the blocks, can be implemented by special purpose hardware-based computer that perform the specified functions, or combinations of special purpose hardware and computer instructions. In addition to the description of <FIG> below, the descriptions of <FIG> are also applicable to <FIG>, and are incorporated herein by reference. Thus, the above description may not be repeated here.

Referring to <FIG>, in an example, operations <NUM>, <NUM>, <NUM> and <NUM> are performed in a neural network of an object recognition apparatus, and operations <NUM>, <NUM> and <NUM> are performed by a processor of the object recognition apparatus. In an example, the dashed lines enclosing operations <NUM>, <NUM>, <NUM>, <NUM> and <NUM> in <FIG> correspond to operations that are performed in parallel in the neural network and the processor.

In operation <NUM>, the object recognition apparatus acquires an input image from an image sensor. The object recognition apparatus provides the input image to the neural network and the processor.

In operation <NUM>, the neural network extracts a feature from the input image. In operation <NUM>, the neural network generates a feature map based on the extracted feature. In parallel with operations <NUM> and <NUM>, in operation <NUM>, the processor extracts an ROI (for example, a road region) corresponding to an object of interest from the input image. In an example, the object recognition apparatus detects an ROI (for example, a road region) including an object of interest (for example, a road) using a separate neural network that is trained to detect the object of interest. In this example, the separate neural network is a neural network trained to detect an ROI including a feature portion of an object of interest together with the object of interest. In another example, the object recognition apparatus detects an ROI (for example, a road region) based on sensor information acquired using a light detection and ranging (LiDAR) sensor as well as an external image captured by a camera or an image sensor. In an example, the sensor information includes, for example, depth information indicating a distance to an object of interest detected from an external image.

In operation <NUM>, the processor calculates an area of the extracted road region. The area of the road region is calculated based on, for example, a ratio of a size of the road region to a size of the input image as described above in Equation <NUM>. For example, it is assumed that the input image has a size of <NUM><NUM>, that the road region in the input image has a size of <NUM><NUM>, and that "<NUM>" object candidate regions for the input image are set as a default in the neural network. In this example, the area of the road region corresponds to <NUM>% of the size of the input image. In operation <NUM>, the processor determines a number of object candidate regions by multiplying a ratio (for example, <NUM>%) of the area of the road region to the size of the input image by a number (for example, "<NUM>") of object candidate regions for the input image that is set as a default in the neural network. For example, "<NUM>" object candidate regions are determined by multiplying a ratio of <NUM>% and "<NUM>.

In an example, the processor stores, in advance in a form of a lookup table, the number of object candidate regions determined based on the ratio of the size of the road region to the size of the input image. For example, when the area of the road region is calculated, the processor determines the number of object candidate regions based on the lookup table. The processor transfers the number of object candidate regions to the neural network.

In operation <NUM>, the neural network determines positions of the object candidate regions on the feature map generated in operation <NUM>, in response to receiving the number of object candidate regions determined in operation <NUM>.

In operation <NUM>, the neural network recognizes an object of interest from the ROI based on the positions of the object candidate regions. Thus, the neural network reduces the time for object recognition, and recognizes the object of interest from the ROI rather than the entire region of the input image.

<FIG> illustrates another example of an object recognition method. The operations in <FIG> may be performed in the sequence and manner as shown, although the order of some operations may be changed or some of the operations omitted without departing from the scope of the illustrative examples described. Many of the operations shown in <FIG> may be performed in parallel or concurrently. One or more blocks of <FIG>, and combinations of the blocks, can be implemented by special purpose hardware-based computer that perform the specified functions, or combinations of special purpose hardware and computer instructions. In addition to the description of <FIG> below, the descriptions of <FIG> are also applicable to <FIG>, and are incorporated herein by reference. Thus, the above description may not be repeated here.

Referring to <FIG>, in operation <NUM>, an object recognition apparatus extracts an ROI corresponding to at least one object of interest from an input image. For example, the object recognition apparatus extracts the ROI using any one or any combination of a training-based scene segmentation algorithm and an image processing algorithm. The object of interest includes, for example, a vehicle, a human, an animal, a plant or a building. Also, the ROI is a region corresponding to a road, optionally in combination with any of a vehicle, a human, an animal, a plant and a building.

In operation <NUM>, the object recognition apparatus determines a number of object candidate regions that are used to detect the object of interest based on a size of the ROI. The object recognition apparatus calculates a ratio of the size of the ROI to a size of the input image and determines a number of object candidate regions based on the calculated ratio as described above in Equation <NUM>.

In operation <NUM>, the object recognition apparatus recognizes the object of interest from the ROI based on the number of object candidate regions in a neural network. The object recognition apparatus determines positions of the object candidate regions on a feature map generated in the neural network, based on the determined number of object candidate regions. The object recognition apparatus recognizes the object of interest from the ROI based on the positions of the object candidate regions. The neural network includes an R-CNN including an RPN and a detection network.

The object recognition apparatus determines a control parameter used to control a speed of a vehicle and a traveling direction of the vehicle based on an object recognition result, and controls traveling of the vehicle using the control parameter.

<FIG> illustrates an example of a process of sequentially performing operations of an object recognition method. The operations in <FIG> may be performed in the sequence and manner as shown, although the order of some operations may be changed or some of the operations omitted without departing from the scope of the illustrative examples described. Many of the operations shown in <FIG> may be performed in parallel or concurrently. One or more blocks of <FIG>, and combinations of the blocks, can be implemented by special purpose hardware-based computer that perform the specified functions, or combinations of special purpose hardware and computer instructions. In addition to the description of <FIG> below, the descriptions of <FIG> are also applicable to <FIG>, and are incorporated herein by reference. Thus, the above description may not be repeated here.

Referring to <FIG>, in an example, operations <NUM>, <NUM>, <NUM> and <NUM> are performed by a processor of an object recognition apparatus, and operations <NUM>, <NUM>, <NUM> and <NUM> are performed by a neural network of the object recognition apparatus.

In operation <NUM>, the object recognition apparatus acquires an input image from a camera or an image sensor. The object recognition apparatus provides the input image to the processor.

In operation <NUM>, the processor extracts an ROI corresponding to a road region corresponding to an object of interest from the input image. The processor transfers the input image and information about the extracted road region to the neural network. In operation <NUM>, the neural network extracts a feature from the input image and the information about the road region. In operation <NUM>, the neural network generates a feature map based on the extracted feature.

The object recognition apparatus detects an road region including an object of interest using a separate neural network that is trained to detect the object of interest. In this example, the separate neural network is a neural network trained to detect an ROI including a feature portion of an object of interest together with the object of interest. In an example, the object recognition apparatus detects an ROI (for example, a road region) based on sensor information acquired using a LiDAR sensor as well as an external image captured by a camera or an image sensor. In this example, the sensor information includes, for example, depth information indicating a distance to an object of interest detected from an external image.

In operation <NUM>, the processor calculates an area of the extracted road region. The area of the road region is calculated based on a ratio of a size of the road region to a size of the input image as described above in Equation <NUM>.

In operation <NUM>, the processor determines a number of object candidate regions by multiplying a ratio corresponding to the area of the road region by a default number of object candidate regions for the input image set in the neural network.

In operation <NUM>, the neural network determines positions of the object candidate regions on the feature map generated in operation <NUM> in response to receiving the number of object candidate regions determined in operation <NUM> from the processor.

In operation <NUM>, the neural network recognizes the object of interest from the ROI based on the positions of the object candidate regions.

<FIG> illustrates an object recognition apparatus <NUM>. Referring to <FIG>, the object recognition apparatus <NUM> includes a sensor <NUM>, a processor <NUM>, and a neural network <NUM>, and may for example include a display <NUM>. The sensor <NUM>, the processor <NUM>, the neural network <NUM>, and the display <NUM> communicate with each other via a communication bus <NUM>. The object recognition apparatus <NUM> may for example further include a memory (not shown).

The sensor <NUM> acquires an input image. The sensor <NUM> includes, for example, an image sensor or a LiDAR sensor. A single sensor <NUM>, or a plurality of sensors <NUM> may be provided.

The processor <NUM> extracts an ROI corresponding to at least one object of interest, in parallel with a generation of a feature map in the neural network <NUM>. Also the processor <NUM> determines, based on a size of the ROI, a number of object candidate regions that are used to detect an object of interest. The processor <NUM> calculates a ratio of the size of the ROI to a size of the input image and determines the number of object candidate regions based on the calculated ratio.

The neural network <NUM> is a faster R-CNN. The neural network <NUM> includes a convolution network <NUM>, an RPN <NUM>, and a detection network <NUM>. The convolution network <NUM> extracts a feature from an input image and generates a feature map. The convolution network <NUM> includes a plurality of convolutional layers. The RPN <NUM> determines an object candidate region for all regions of an input image. A number of determined object candidate regions correspond to a default value. The detection network <NUM> recognizes an object of interest from an ROI based on the number of object candidate regions. The convolution network <NUM>, the RPN <NUM> and the detection network <NUM>, respectively, correspond to the CNN <NUM>, the RPN <NUM> and the detection network that includes the ROI pooling layer <NUM> and the classifier <NUM> of <FIG>, and accordingly the above description of <FIG> is applicable to an operation of each of the convolution network <NUM>, the RPN <NUM> and the detection network <NUM>.

In an example, the object recognized by the object recognition apparatus <NUM> is output to a display <NUM>. In an example, the object recognition apparatus <NUM> displays the object on a windshield glass of the vehicle through a head-up display (HUD). However, the displaying of the position is not limited to the example described in the forgoing, and any other instrument cluster, vehicular infotainment system, screen in the vehicle that uses augmented reality, or display panel in the vehicle may perform the display function. Other displays, such as, for example, smart phone and eye glass display (EGD) that are operatively connected to the object recognition apparatus <NUM> may be used without departing from the scope of the illustrative examples described.

The memory stores the input image, and the number of object candidate regions determined by the processor <NUM>. In an example, the memory stores a lookup table in which a number of object candidate regions based on the ratio of the size of the ROI to the size of the input image are stored in advance. The memory is, for example, a volatile memory or a nonvolatile memory. Further description of the memory is provided below.

The processor <NUM> extracts an ROI corresponding to at least one object of interest from the input image, and determines, based on the size of the ROI, a number of object candidate regions that are used to detect the object of interest. In this example, the neural network <NUM> includes a plurality of layers configured to recognize the object of interest from the ROI based on the number of object candidate regions.

In an example, the processor <NUM> and the neural network <NUM> perform at least one of the methods described above with reference to <FIG>, or an algorithm corresponding to at least one of the methods. The processor <NUM> and the neural network <NUM> execute a program and control the object recognition apparatus <NUM>. Program codes executed by the processor <NUM> and the neural network <NUM> are stored in the memory.

The object recognition apparatus <NUM>, the neural network <NUM>, other apparatuses, units, modules, devices, and components described herein with respect to <FIG>, <FIG> and <FIG> are implemented by hardware components. Examples of hardware components that may be used to perform the operations described in this application where appropriate include controllers, sensors, generators, drivers, memories, comparators, arithmetic logic units, adders, subtractors, multipliers, dividers, integrators, and any other electronic components configured to perform the operations described in this application. In other examples, one or more of the hardware components that perform the operations described in this application are implemented by computing hardware, for example, by one or more processors or computers. A processor or computer may be implemented by one or more processing elements, such as an array of logic gates, a controller and an arithmetic logic unit, a digital signal processor, a microcomputer, a programmable logic controller, a field-programmable gate array, a programmable logic array, a microprocessor, or any other device or combination of devices that is configured to respond to and execute instructions in a defined manner to achieve a desired result. In one example, a processor or computer includes, or is connected to, one or more memories storing instructions or software that are executed by the processor or computer. Hardware components implemented by a processor or computer may execute instructions or software, such as an operating system (OS) and one or more software applications that run on the OS, to perform the operations described in this application. The hardware components may also access, manipulate, process, create, and store data in response to execution of the instructions or software. For simplicity, the singular term "processor" or "computer" may be used in the description of the examples described in this application, but in other examples multiple processors or computers may be used, or a processor or computer may include multiple processing elements, or multiple types of processing elements, or both. For example, a single hardware component or two or more hardware components may be implemented by a single processor, or two or more processors, or a processor and a controller. One or more hardware components may be implemented by one or more processors, or a processor and a controller, and one or more other hardware components may be implemented by one or more other processors, or another processor and another controller. One or more processors, or a processor and a controller, may implement a single hardware component, or two or more hardware components. A hardware component may have any one or more of different processing configurations, examples of which include a single processor, independent processors, parallel processors, single-instruction single-data (SISD) multiprocessing, single-instruction multiple-data (SIMD) multiprocessing, multiple-instruction single-data (MISD) multiprocessing, and multiple-instruction multiple-data (MIMD) multiprocessing.

Instructions or software to control a processor or computer to implement the hardware components and perform the methods as described above are written as computer programs, code segments, instructions or any combination thereof, for individually or collectively instructing or configuring the processor or computer to operate as a machine or special-purpose computer to perform the operations performed by the hardware components and the methods as described above. In one example, the instructions or software includes at least one of an applet, a dynamic link library (DLL), middleware, firmware, a device driver, an application program storing the method of preventing the collision.

The instructions or software to control a processor or computer to implement the hardware components and perform the methods as described above, and any associated data, data files, and data structures, are recorded, stored, or fixed in or on one or more non-transitory computer-readable storage media. Examples of a non-transitory computer-readable storage medium include read-only memory (ROM), random-access programmable read only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random-access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROMs, CD-Rs, CD+Rs, CD-RWs, CD+RWs, DVD-ROMs, DVD-Rs, DVD+Rs, DVD-RWs, DVD+RWs, DVD-RAMs, BD-ROMs, BD-Rs, BD-R LTHs, BD-REs, blue-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), flash memory, a card type memory such as multimedia card micro or a card (for example, secure digital (SD) or extreme digital (XD)), magnetic tapes, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, solid-state disks, and any other device that is configured to store the instructions or software and any associated data, data files, and data structures in a non-transitory manner and providing the instructions or software and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the instructions.

Claim 1:
An object recognition method comprising:
acquiring (<NUM>) an input image from an image sensor;
extracting (<NUM>) a road region from the input image using a scene segmentation algorithm;
calculating (<NUM>) a ratio of an area of the extracted road region to a size of the input image;
determining (<NUM>) what number of object candidate regions is to be used to detect an object of interest by applying the ratio to a default number of object candidate regions;
in a faster region-based convolutional neural network, R-CNN (<NUM>), comprising a convolutional neural network (<NUM>), a region proposal network, RPN (<NUM>), and a detection network:
extracting (<NUM>) in the convolutional neural network, a feature from the input image and generating (<NUM>) a feature map based on the extracted feature;
determining (<NUM>), in the RPN, positions of object candidate regions on the feature map, based on the determined number of object candidate regions; and
recognizing (<NUM>), in the detection network, the object of interest from only the extracted road region, based on the determined positions of the object candidate regions.