Patent Description:
In transportation systems, the issue of ensuring driving safety and driver assistance is an important requirement. One of the factors which are responsible for the road accidents is either driver's ignorance or reduced range of vision. Conventional safety features like seat belts, airbags, Anti-lock braking system (ABS) are available for reducing severity of the accidents. Further, other safety features available today include driver-assistance systems helping the driver to avoid accidents by providing early alerts to the driver and if required taking over the control of a vehicle from the driver.

One of such driver-assistance systems includes collision avoidance system enabled for detecting objects in the path of the vehicle and alerting the driver. For detecting the objects, images are captured and further processed for detecting an actual position of the objects from the vehicle. However, present techniques available for providing the collision avoidance system are capable of detecting the objects, one at a time, either placed at a far range or at a near range from the vehicle. Thus, the present driver-collision avoidance system face technical challenge of detecting the objects placed at the far range and the near range from the vehicle simultaneously. Additionally, the computation of objects, by the existing systems, at different ranges (far and near) requires more computational time which may further lead in delaying the response provided in form of notification alerts to the driver.

Document <CIT> discloses a vehicular pedestrian detection device efficiently deciding a pedestrian without being affected by information except a shape such as a pattern or a wrinkle of clothing of the pedestrian. ;SOLUTION: In this vehicular pedestrian detection device, a distance image creation means <NUM> creates a distance image F3 removed with information not needing analysis related to a luminance change, and a normalization means <NUM> and a deciding cell setting means <NUM> set a deciding cell S(n, m) of a smaller range than the distance image F3. A first distance gradient vector calculation means <NUM> and a first histogram creation means <NUM> create histogram data H1 necessary for pedestrian detection about a distance gradient vector V<SB>K</SB>of the deciding cell S(n, m) as a specific area corresponding to a shape of an object O. Because comparing the histogram data H1 with histogram reference data H0, the vehicular pedestrian detection device can decide the pedestrian in a state that the information not needing the analysis in the shape decision to speed up pedestrian decision processing and to prevent false detection.

Document "Vehicle detection combining gradient analysis and AdaBoost classification" discloses a real-time vision-based vehicle's rear detection system using gradient based methods and Adaboost classification, for ACC applications. The detection algorithm consists of two main steps: gradient driven hypothesis generation and appearance based hypothesis verification. In the hypothesis generation step, possible target locations are hypothesized. This step uses an adaptive range-dependant threshold and symmetry for gradient maxima localization. Appearance-based hypothesis validation verifies those hypothesis using AdaBoost for classification with illumination independent classifiers. The monocular system was tested under different traffic scenarios (e.g., simply structured highway, complex urban environments, varying lightening conditions), illustrating good performance.

Document <CIT> discloses a face detecting unit that detects a person's face from input image data, and a parameter setting unit sets parameters for generating a gradient histogram indicating the gradient direction and gradient magnitude of a pixel value based on the detected face. Further, a generating unit sets a region (a cell) from which to generate a gradient histogram in the region of the detected face, and generates a gradient histogram for each such region to generate feature vectors. An expression identifying unit identifies an expression exhibited by the detected face based on the feature vectors. Thereby, the facial expression of a person included in an image is identified with high precision.

This summary is provided to introduce aspects related to devices and methods for detecting objects at multiple ranges and the concepts are further described below in the detailed description. This summary is not intended to identify essential features of disclosure nor is it intended for use in determining or limiting the scope of the disclosure.

The invention is set out in the appended set of claims <NUM>-<NUM>.

The same numbers are used throughout the drawings to refer like features and components.

Devices and methods for detecting objects at multiple ranges simultaneously on a path of a vehicle are described. While driving on the road, different objects like vehicle, pedestrian, an animal or any other object may come in front of the vehicle. For driving safely, it is required to detect the position of these objects before it get too close to the vehicle. According to embodiments of present disclosure, one or more onboard cameras or image capturing units installed on the vehicle captures the images of the objects appearing at multiple ranges from the vehicle. The objects is placed at a near distance and far distance from the vehicle. The device disclosed in the present disclosure is capable for simultaneously detecting the objects placed at the near distance and at the far distance, thus providing the multi-range object detection.

The image of the objects captured by an image capturing unit is processed by the device at different stages. The input image frame captured by the unit is first split into multiple images slices of images indicating region of interests (ROI). The processing is performed upon the multiple images in order to reduce the noise, isolate the individual elements, join the disconnected parts, sharpen the edges, and smoothening the multiple images using smoothening filters. After processing the images through the different stages, one or more features are extracted. Based on the one or more features, gradient is computed simultaneously corresponding to each of these multiple images. Further, cell histograms is created based on values associated with the gradient. The cell histogram is created by casting weighted vote for an orientation based histogram channel. Further, the gradients computed is normalized by using normalizing factor in order to reduce the effect of changes in illumination and contrast from the images. After the normalization, a Support Vector Machine (SVM) linear classifier is deployed for simultaneously classifying the near object and the far object in a category of a pedestrian or a vehicle. Thus, different objects placed at multiple ranges (near objects and far objects) from the vehicle is detected simultaneously by the device. The detection of the objects at multiple ranges is used for vehicle's safety.

While aspects of described device and method for detecting objects at multiple ranges simultaneously is implemented in any number of different computing devices, environments, and/or configurations, the embodiments are described in the context of the following exemplary devices.

Referring to <FIG>, a network implementation <NUM> of a server <NUM> and a device <NUM> for simultaneously detecting the objects at multiple ranges is illustrated, in accordance with an embodiment of the present disclosure. In one embodiment, the device <NUM> facilitates the detection of the one or more objects in a path of a vehicle <NUM>. Although the present disclosure is explained considering that the server <NUM> is implemented as a computing system, it is understood that the server <NUM> is also implemented as a variety of computing systems, such as a laptop computer, a desktop computer, a notebook, a workstation, a mainframe computer, a server, a network server, a tablet, a mobile phone, and the like. In one implementation, the server <NUM> is implemented in a cloud-based environment. According to an embodiment, the device <NUM> is implemented with the vehicle <NUM>. Further, the server <NUM> tracks the activities of the device <NUM>, and the device <NUM> is communicatively coupled to the server <NUM> through a network <NUM>.

In one implementation, the network <NUM> is a wireless network, a wired network or a combination thereof. The network <NUM> is implemented as one of the different types of networks, such as intranet, local area network (LAN), wide area network (WAN), the internet, and the like. The network <NUM> is either a dedicated network or a shared network. The shared network represents an association of the different types of networks that use a variety of protocols, for example, Hypertext Transfer Protocol (HTTP), Transmission Control Protocol/Internet Protocol (TCP/IP), Wireless Application Protocol (WAP), and the like, to communicate with one another. Further the network <NUM> includes a variety of network devices, including routers, bridges, servers, computing devices, storage devices, and the like.

Referring now to <FIG>, the device <NUM> is illustrated in accordance with an embodiment of the present disclosure. In one embodiment, the device <NUM> includes at least one processor <NUM>, an input/output (I/O) interface <NUM>, and a memory <NUM>. The at least one processor <NUM> is implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, the at least one processor <NUM> is configured to fetch and execute computer-readable instructions or modules stored in the memory <NUM>.

The I/O interface <NUM> includes a variety of software and hardware interfaces, for example, a web interface, a graphical user interface, and the like. The I/O interface <NUM> allows the device <NUM> to interact with a user directly or through the client devices <NUM>. Further, the I/O interface <NUM> enables the device <NUM> to communicate with other computing devices, such as web servers and external data servers (not shown). The I/O interface <NUM> facilitates multiple communications within a wide variety of networks and protocol types, including wired networks, for example, LAN, cable, etc., and wireless networks, such as WLAN, cellular, or satellite. The I/O interface <NUM> includes one or more ports for connecting a number of devices to one another or to another server.

The memory <NUM> includes any computer-readable medium or computer program product known in the art including, for example, volatile memory, such as static random access memory (SRAM) and dynamic random access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, a compact disks (CDs), digital versatile disc or digital video disc (DVDs) and magnetic tapes. The memory <NUM> includes modules <NUM> and data <NUM>.

The modules <NUM> include routines, programs, objects, components, data structures, etc., which perform particular tasks or implement particular abstract data types. In one implementation, the modules <NUM> includes a receiving module <NUM>, a splitting module <NUM>, detecting module <NUM>, an image processing module <NUM>, and other modules <NUM>. The other modules <NUM> includes programs or coded instructions that supplement applications and functions of the device <NUM>.

The data <NUM>, amongst other things, serves as a repository for storing data processed, received, and generated by one or more of the modules <NUM>. The data <NUM> also includes an image database <NUM>, and other data <NUM>.

Referring now to <FIG> illustrates detailed working of the device <NUM>, in accordance with an embodiment of the present disclosure. The device <NUM> disclosed in the present disclosure is implemented with any type of vehicle. In a present scenario, a four-wheeler vehicle is shown (<FIG>) i.e., vehicle <NUM>. The device <NUM> is provided for detecting objects at a near and far distance (i.e., multiple ranges) from the vehicle <NUM> in order to avoid collision of the vehicle <NUM> with the object. While the vehicle <NUM> is in the motion, it is extremely important to accurately detect the objects in the path of the vehicle <NUM>. The objects appear at near as well as far distance from the vehicle <NUM>. Further, the objects is in a still or a moving position. In one example, object <NUM> is placed at a far distance and object <NUM> is placed at near distance from the vehicle <NUM> as shown in <FIG>. Thus, at the same time the detection of the objects placed at the far distance (object <NUM>) as well as the near distance (object <NUM>) from the vehicle <NUM> is required. The device <NUM> of the present disclosure facilitates a simultaneous detection of the objects at the near and the far distance from the vehicle <NUM>. Further, according one embodiment of present disclosure, the far distance ranges up to <NUM> meters, and the near distance ranges up to <NUM> meters. It must be noted to a person skilled in art that the ranges corresponding to near and far distance from the vehicle <NUM> varies and the present disclosure is not limited to aforementioned ranges of the far distance and the near distance.

According to embodiments of present disclosure, an image of the object <NUM> and the object <NUM> is captured by an image capturing unit <NUM> of the vehicle <NUM>. The image capturing unit <NUM> is a mono or stereo type camera (pair of imagers) used for capturing the images of one or more objects placed at the near and the far distance from the vehicle <NUM>. The images captured is stored in an image database <NUM> of the device <NUM>. The receiving module <NUM> of the device <NUM> receives the image corresponding to the objects appearing on the path of the vehicle <NUM>. At first, the splitting module <NUM> of the device <NUM> splits the image received into a plurality of sub-images/slices indicating region of interest (ROIs). Further, each sub-image of the plurality of sub-images is in a form of rectangular windows of pixels computed based on distance of the detection (i.e., distance between the objects and the vehicle <NUM>). Since the objects looks bigger in short distance range and smaller in long distance range, there is a big rectangular window corresponding to short range of distance, and a smaller rectangular window corresponding to long range of distance. In one exemplary embodiment, a range for the region of interest (ROI) is defined such that a sub-image located at a distance up to <NUM> meters, <NUM> meters <NUM> meters, from the vehicle <NUM>, is sliced into ROI1, ROI2, and ROI3 respectively. The above example of the range defined for the ROIs is applicable for VGA type camera. It is to be noted to a person skilled in art that, the distance of detection (i.e., ranges defined for ROIs) improves with resolution of the camera and further the ROI selection varies accordingly.

Since, the objects at the longer distance, for example <NUM> meters, are perceivably smaller in size, it becomes difficult to detect the object image at the original scale. The image processing module <NUM> of the device <NUM> enlarges one or more sub-images of the plurality of sub-images based on the distance of the near object and the far object from the vehicle. According to embodiments of present disclosure, the one or more sub-images are enlarged in size by using a bilinear interpolation method to 2x times the original of the ROI2, and is 3x times the original of the ROI3. After interpolation of the one or more sub-images, the image processing module <NUM> applies erosion followed by dilation upon the interpolated one or more sub-images. The application of the erosion and the dilation results in reduction in a noise of the one or more sub-images, isolation of individual elements, and joining of disconnected parts in the one or more sub-images. Further, the dilation involves convoluting of the one or more sub-images with a kernel which is a circle. The kernel is scanned over the one or more sub-images and finds the maximum pixel value overlapped by the kernel and replaces the pixel value with the maximum pixel value. The dilation does the opposite of this as it tries to find local minimum over the area of kernel. With this the bright areas in the one or more sub-images gets thinner whereas the dark areas get thicker.

Upon applying the erosion and the dilation, the image processing module <NUM> further sharpens the one or more sub-images with a two dimensional image sharpening filter with a median of 3x3 kernels. This step helps in sharpening the edges while removing the noise from the one or more sub-images. According to an embodiment of the present disclosure, the kernel used as the image sharpening filter is as follows: <MAT> Where the edgex = window width/<NUM>; and the edgey = window height/<NUM>.

Subsequently, the image processing module <NUM> is further configured for smoothening the one or more sub-images using a two dimensional smoothening filters with 3x3 Gaussian kernel. Using the Gaussian kernel helps in reducing the blur from the one or more sub-images which is introduced due to motion of the image capturing unit <NUM>. According to embodiments of present disclosure, the Gaussian function used is as below: <MAT> Where x is the distance from the origin in the horizontal axis and y is the distance from the origin in the vertical axis and α is the standard deviation.

Subsequent to smoothening of the one or more sub-images and reducing the blur from the one or more sub-images, the detecting module <NUM> of the device <NUM> detects a near object and a far object from the plurality of sub-images by extracting one or more features from the plurality of sub-images. The detecting module <NUM> further simultaneously processes each of the plurality of sub-images for computing gradient associated with the each of the plurality of sub-images based on the one or more features extracted. In one embodiment, three different window sizes is processed simultaneously for computing the gradient associated with three different ROIs (ROI1, ROI2, and ROI3). According to embodiments, a one dimensional derivate mask is scanned in both horizontal and vertical directions. Further, the intensity channel of the image data is filtered using the following kernel: <MAT>.

Further, the detecting module <NUM> creates cell histograms by casting a weighted vote for an orientation based histogram channel based on the values associated with the gradient computed. The cell histograms comprises plurality of cells rectangular in shape and the histogram of channels are spaced over <NUM> degrees with unsigned gradient with nine histogram channels.

Further, the detecting module <NUM> of the device <NUM> normalizes the gradient strengths in order to reduce the effect of changes in illumination and contrast. For normalizing the gradient strengths, the cells of the plurality of cells is grouped in spatial blocks. According to embodiments of the present disclosure, for each of the ROIs (sub-images) for which the gradient is computed, gradient value is divided into window size which is specific to the ROI. In one exemplary example, for the ROI1, <NUM> x <NUM> is used and gradient values is divided into <NUM>×<NUM> pixel block which in turn contains <NUM> cells of 8x8 pixels and each such cell in turn contain <NUM> histogram bin values. So finally <NUM> weighted bin values are computed which gets multiplied with <NUM> blocks creating <NUM> descriptor values. Thus, doing this for each of the ROIs and window size in the way as discussed above, the detecting module <NUM> significantly improves the speed of processing than the traditional way of computing the gradient for each window separately Further, by processing the three different window sizes simultaneously for computing the gradient associated with the three different ROIs, a concept of an enhanced histogram of oriented gradients (EHOG) is disclosed as per the present disclosure. Thus, the EHOG overcomes the limitation of Histogram of Oriented Gradients (HOG) i.e., processing only one window size at a time by processing the three different window sizes simultaneously. Further, the block normalization is performed by the detecting module <NUM> using following normalizing factor: <MAT>.

After the normalization, the detecting module <NUM> of the device <NUM> applies a Support Vector Machine (SVM) linear classifier on the plurality of sub-images in order to classify the near object and the far object in a category of a pedestrian or a vehicle. According to an embodiment, a SVM learning is performed on samples containing images of objects of interest (for eg: vehicles) cropped to the size of the window sizes as discussed above. Further, the detecting module <NUM> uses following formulae for efficiently classifying the one or more objects images simultaneously with each of the window size and each of the ROI. <MAT> Where Xi is the feature vector of Histogram of Oriented Gradients (HOG), Yi is the feature vector of SVM trained data, N is number of samples per block, M1 is the number of blocks per window for 128x64, and M2 for 64x64 and M3 for 32x32 windows. The SVM trained data comprises predefined features associated with the objects categorized into predefined categories. When the objects (of which the image is captured by the image capturing unit <NUM>) has to be detected and classified into the category, the SVM classifier compares the one or more features of the objects with the predefined features of the SVM trained data. Based on the comparison, the objects are detected as the near and the far object as well as the objects are categorized into the category. As with the above formula, <NUM> different scale windows are simultaneously processed for classification, it ensures multi scale classification of objects into the category (eg: pedestrian, vehicle) without major overhead on the computational complexity. It is to be noted to a person skilled in art, that the objects are classified into other categories (other than pedestrian or vehicle) based on the predefined features of the SVM trained data. For example, the predefined features for the category "pedestrian" are hands, eyes, legs and the like. Similarly, the predefined features for the category "vehicle" are number plate, wheels, steering and the like.

Referring now to <FIG>, method for detecting objects at multiple ranges simultaneously on a path of a vehicle is shown, in accordance with an embodiment of the present disclosure. The method <NUM> is described in the general context of computer executable instructions. Generally, computer executable instructions includes routines, programs, objects, components, data structures, procedures, modules, functions, etc., that perform particular functions or implement particular abstract data types. The method <NUM> is also be practiced in a distributed computing environment where functions are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, computer executable instructions are located in both local and remote computer storage media, including memory storage devices.

The order in which the method <NUM> is described is not intended to be construed as a limitation, and any number of the described method blocks is combined in any order to implement the method <NUM> or alternate methods. Furthermore, the method is implemented in any suitable hardware, software, firmware, or combination thereof. However, for ease of explanation, in the embodiments described below, the method <NUM> is considered to be implemented in the above described device <NUM>.

At block <NUM>, image corresponding to objects appearing on the path of the vehicle is received. The image received is captured by an image capturing unit installed on the vehicle.

At block <NUM>, the image received is split into plurality of sub-images indicating region of interest (ROIs). Further, each of the plurality of sub-images is in a form of a rectangular window of pixels computed based on distance of the objects from the vehicle. The plurality of sub-images is further processed for reducing the noise from the images, isolating the individual elements from the images, joining the disconnected parts of the images, sharpening the edges of the images, and further smoothening the images using smoothening filters.

At block <NUM>, the near and far object from the plurality of sub-images is detected performing the steps shown in blocks 406A-406E.

At block 406A, one or more features is extracted from the plurality of sub-images.

At block 406B, each of the plurality of sub-images is simultaneously processed for computing gradient associated with each of the plurality of sub-images based on the one or more features extracted.

At block 406C, cell histograms comprising plurality of cells is created by casting weighted vote for an orientation based histogram channel based on values associated with the gradient.

At block 406D, the gradients computed is normalized by grouping the cells of the plurality of cells in spatial blocks in order to normalize the plurality of sub-images.

At block 406E, a Support vector Machine (SVM) linear classifier is applied on the plurality of sub-images after being normalized in order to detect and classify the near object and the far object in a category.

Claim 1:
A method for detecting objects at multiple ranges simultaneously on a path of a vehicle, the method comprising:
receiving, by a processor, an image as an input corresponding to objects appearing on the path of the vehicle;
splitting, by the processor, the image into a plurality of sub-images indicating region of interest (ROIs), wherein each of the plurality of sub-images is in a form of a rectangular window of pixels computed based on distance of the objects from the vehicle, wherein the distance of the objects from the vehicle is detected based on resolution of the at least one image capturing unit, and wherein a big rectangular window of pixels indicate that the objects are at a short range of distance from the vehicle, and a small rectangular window of pixels indicate that the objects are at a long range of distance from the vehicle; and
detecting, by the processor, a first object and a second object at different ranges from the plurality of sub-images by extracting one or more features from the plurality of sub-images;
simultaneously processing each of the plurality of sub-images for computing gradient associated with the each of the plurality of sub-images based on the one or more features extracted, wherein processing each of the plurality of sub-images includes:
interpolating the one or more sub-images based on the distance of the first object and the second object from the vehicle, wherein the interpolation of the one or more sub-images comprises enlarging size of the one or more sub-images using a bilinear interpolation technique; and
applying erosion and dilation upon the one or more sub-images in order to reduce noise from the one or more sub-images, isolating individual elements of the one or more sub-images, and joining of disconnected parts in the one or more sub-images,
creating cell histograms comprising plurality of cells by casting weighted vote for an orientation based histogram channel based on values associated with the gradient computed,
normalizing the gradients computed by grouping the cells of the plurality of cells in spatial blocks in order to normalize the plurality of sub-images;
and
applying a Support vector Machine (SVM) linear classifier on the plurality of sub-images after being normalized in order to detect and classify the first object and the second object in a category.