Patent Description:
Digital imaging processing, for example in automotive applications, may be a computationally expensive task.

Accordingly, there is a need to increase efficiency of digital image processing.

<CIT> discloses selecting a threshold determination method from among a plurality of alternative global thresholding determination methods and, optionally, a local thresholding determination method based on characteristics of a histogram of grayscales values representing an image. When it is determined to use a global thresholding method, a single global binarization threshold value is determined using the selected global thresholding method. When it is determined to use local thresholding, a plurality of local binarization threshold values are determined corresponding to different non-overlapping blocks of the image. The determined binarization threshold(s) are applied to the gray scale pixel values to obtain a set of binary pixel values.

The present disclosure provides a computer implemented method, a computer system, a vehicle, and a non-transitory computer readable medium according to the independent claims. Embodiments are given in the subclaims, the description and the drawings.

In one aspect, the present disclosure is directed at a computer implemented method for filtering portions of an image, the method comprising the following steps performed (in other words: carried out) by computer hardware components: dividing the image into a plurality of segments, each segments comprising a plurality of pixels; for each of the segments, determining at least one of an expected value, a standard deviation, and a kurtosis of the plurality of pixels of the respective segment; clustering the plurality of segments into a plurality of clusters based on the at least one of the expected value, the standard deviation, and the kurtosis of the plurality of pixels of the respective segment; for each of the clusters, determining the respective cluster as belonging to a background based on a size of the respective cluster; and determining a filtered image based on the background.

The filtered image may include only portions of the image not belonging to the background.

The segments may be rectangular portions of the image. The segments may be non-overlapping (in other words: each pixel may belong to exactly one segment). The plurality of segments, taken all together, may cover the entire image (in other words: each pixel may belong to at least one segment).

According to another aspect, the at least one of an expected value, a standard deviation, and a kurtosis are determined based on gray values of the respective pixels. For example, for a color image, the image may be converted into a gray scale image before dividing it into the plurality of segments.

According to another aspect, the computer implemented method further comprises the following step carried out by the computer hardware components: recovering segments from clusters belonging to the background as not belonging to the background. Recovering may be understood as determining, for a cluster belonging to the background, a segment of the cluster as not belonging to the background.

According to another aspect, segments from clusters belonging to the background are recovered based on a kurtosis of an expected value for each segment of the respective cluster. In other words, for each segment of the respective cluster, an expected value may be determined, and a kurtosis of these expected values may be determined and used for the recovering.

According to another aspect, segments from clusters belonging to the background are recovered based on a comparison of the kurtosis of the expected value for each segment of cluster with a kurtosis of the respective segment individually.

According to another aspect, the computer implemented method further comprises the following step carried out by the computer hardware components: providing an output comprising an indication for each pixel whether the pixel belongs to the background. A pixel may belong to the background if the segment to which the pixel belongs belongs to the background.

According to another aspect, the computer implemented method further comprises the following step carried out by the computer hardware components: providing an output comprising an indication for each segment whether the segment belongs to the background.

According to another aspect, the computer implemented method further comprises the following step carried out by the computer hardware components: further processing the image. The further processing of the image may disregard (or not take into account or not process) portions (for example pixels or segments or clusters) of the image belonging to the background.

According to another aspect, the further processing comprises at least one of object detection or object classification.

According to another aspect, pixels belonging to the background and/or segments belonging to the background are excluded from the further processing. This may reduce processing cost, while at the same time, by the determination of the background with the method according to various aspects, the important details of the image may be preserved and thus may be processed.

In another aspect, the present disclosure is directed at a computer system, said computer system comprising a plurality of computer hardware components configured to carry out several or all steps of the computer implemented method described herein.

The computer system may comprise a plurality of computer hardware components (for example a processor, for example processing unit or processing network, at least one memory, for example memory unit or memory network, and at least one non-transitory data storage). It will be understood that further computer hardware components may be provided and used for carrying out steps of the computer implemented method in the computer system. The non-transitory data storage and/or the memory unit may comprise a computer program for instructing the computer to perform several or all steps or aspects of the computer implemented method described herein, for example using the processing unit and the at least one memory unit.

According to another aspect, the computer system further comprises a camera configured to acquire the image.

In another aspect, the present disclosure is directed at a vehicle comprising the computer system as described herein.

According to another aspect, the vehicle further comprises a camera configured to acquire the image.

Digital imaging processing, for example in automotive applications, may be a computationally expensive task. The computational effort of image processing may depend on the number of pixels in the image to be processed.

For example, an image may be scaled to a smaller size, so that the computational effort may be reduced. However, important details in the image may be lost.

Furthermore, it has been found that often a large portion of the pixels of an image does not include details of interest.

According to various embodiments, a method for filtering portions of an image (in other words: a method for filtering an image) may be provided. For example, the image may be filtered so that the details of interest are preserved, but unnecessary information is removed from consideration.

The methods according to various embodiments may provide statistical visual background filtering.

The method may filter unnecessary information from an image (or from a plurality of images, for example from images of a video), for example taken by a camera mounted on a vehicle, by dividing the images into a fixed number of segments for separate processing to allow focusing of later processing, for example classification processing, on crucial parts of the images.

In order to filter unnecessary information from an image (which may also be referred to as a picture), the image may be divided it into a fixed number of segments.

For each segment, the mathematical expectation (M, which may also be referred to as expected value, mean, average, or first moment), the standard deviation (Sigma) and the kurtosis (K) may be calculated based on a grey (resolution) level of pixels.

Defining segment similarity, segments may be clustered using dynamically redefined keys for each cluster and a chosen level of tolerance for incoming segment. For example, a best possible cluster in terms of mathematical expectation may be chosen.

Based on number of created groups (in other words: clusters), a minimal size of a cluster (for example size in terms of segments or in terms of pixels) may be defined for which the cluster may be classified as "background" (in other words: belonging to background; in other words: including unnecessary information).

Each cluster (in other words: group) may have segments and there may be several clusters (groups). A background cluster (in other words: background group) may be one that contains segments, with a segment count exceeding a threshold value. An Initial threshold value may be chosen as N / <NUM>, where N may be a maximum segments count across all similarity clusters (groups). So the cluster size (in other words: group size; in other words: segments count) may provide its "identity" or a candidate being filtered out.

For each cluster classified as (belonging to the) background, a statistic called Kurtosis may be calculated again, but this time on the basis of a mathematical expectation for each segment in a given cluster.

Comparing the individual kurtosis of each segment and the cluster kurtosis, a second phase of filtering may be performed according to various embodiments, but this time it may be decided which segments should be recovered from "background" type of cluster (in other words: which segments which are included in a "background" cluster do not actually belong to the background; in other words: which segments should be moved from "background" to "non-background" or "foreground").

For example, the non-background segments may be desired to include objects like traffic signs and lights.

Filtering unnecessary information from images or a video may allow for classification methods, for example in cars, to focus on the most crucial parts of the image and may provide savings in data storage during software development.

<FIG> shows a flow diagram <NUM> illustrating a method for filtering portions of an image according to various embodiments. At <NUM>, the image may be divided into a plurality of segments, each segments including a plurality of pixels. At <NUM>, for each of the segments, at least one of an expected value, a standard deviation, and a kurtosis of the plurality of pixels of the respective segment may be determined. At <NUM>, the plurality of segments may be clustered into a plurality of clusters based on the at least one of the expected value, the standard deviation, and the kurtosis of the plurality of pixels of the respective segment. At <NUM>, for each of the clusters, the respective cluster may be determined as belonging to a background based on a size of the respective cluster. At <NUM>, a filtered image may be determined based on the background.

The method may provide (or forward) an image (which may be a pixels set) with the original dimension, wherein clusters (in other words: groups) determined as backgrounds are filled with zero volume pixel items. For example, when RGB (red green blue) background clusters (in other words: groups; in other words: segments sets) may be combined from all clusters/groups not depending on their location on an image having pixels with a (<NUM>, <NUM>, <NUM>) three tuple. In other words, in the filtered image, pixels belonging to the background may be set to a value of <NUM>.

For further processing (for example when applying a vision method) to the filtered image, pixel items with zero volume (in other words: zero value) may be ignored.

According to various embodiments, the at least one of an expected value, a standard deviation, and a kurtosis may be determined based on gray values of the respective pixels.

According to various embodiments, the computer implemented method may further include the following step carried out by the computer hardware components: recovering segments from clusters belonging to the background as not belonging to the background.

According to various embodiments, segments from clusters belonging to the background may be recovered based on a kurtosis of an expected value for each segment of the respective cluster.

According to various embodiments, segments from clusters belonging to the background may be recovered based on a comparison of the kurtosis of the expected value for each segment of cluster with a kurtosis of the respective segment individually.

According to various embodiments, the computer implemented method may further include the following step carried out by the computer hardware components: providing an output including an indication for each pixel whether the pixel belongs to the background.

According to various embodiments, the computer implemented method may further include the following step carried out by the computer hardware components: providing an output including an indication for each segment whether the segment belongs to the background.

According to various embodiments, the computer implemented method may further include the following step carried out by the computer hardware components: further processing the image.

According to various embodiments, the further processing may include or may be at least one of object detection or object classification.

According to various embodiments, pixels and/or segments belonging to the background may be excluded from the further processing.

Each of the steps <NUM>, <NUM>, <NUM>, <NUM>, <NUM> and the further steps described above may be performed by computer hardware components.

<FIG> shows an image filtering system <NUM> according to various embodiments. The image filtering system <NUM> may include a dividing circuit <NUM>, an expected value determination circuit <NUM>, a clustering circuit <NUM>, a background determination circuit <NUM>, and a filtered image determination circuit <NUM>.

The dividing circuit <NUM> may be configured to divide the image into a plurality of segments, each segments comprising a plurality of pixels.

The expected value determination circuit <NUM> may be configured to, for each of the segments, determine at least one of an expected value, a standard deviation, and a kurtosis of the plurality of pixels of the respective segment.

The clustering circuit <NUM> may be configured to cluster the plurality of segments into a plurality of clusters based on the at least one of the expected value, the standard deviation, and the kurtosis of the plurality of pixels of the respective segment.

The background determination circuit <NUM> may be configured to for each of the clusters, determine the respective cluster as belonging to a background based on a size of the respective cluster.

The filtered image determination circuit <NUM> may be configured to determine a filtered image based on the background.

The dividing circuit <NUM>, the expected value determination circuit <NUM>, the clustering circuit <NUM>, the background determination circuit <NUM>, and the filtered image determination circuit <NUM> may be coupled with each other, e.g. via an electrical connection <NUM>, such as e.g. a cable or a computer bus or via any other suitable electrical connection to exchange electrical signals.

A "circuit" may be understood as any kind of a logic implementing entity, which may be special purpose circuitry or a processor executing a program stored in a memory, firmware, or any combination thereof.

<FIG> shows a computer system <NUM> with a plurality of computer hardware components configured to carry out steps of a computer implemented method for filtering portions of an image according to various embodiments. The computer system <NUM> may include a processor <NUM>, a memory <NUM>, and a non-transitory data storage <NUM>. A camera <NUM> may be provided as part of the computer system <NUM> (like illustrated in <FIG>), or may be provided external to the computer system <NUM>.

The processor <NUM> may carry out instructions provided in the memory <NUM>. The non-transitory data storage <NUM> may store a computer program, including the instructions that may be transferred to the memory <NUM> and then executed by the processor <NUM>. The camera <NUM> may be used for determining the image (for example the image to be filtered).

The processor <NUM>, the memory <NUM>, and the non-transitory data storage <NUM> may be coupled with each other, e.g. via an electrical connection <NUM>, such as e.g. a cable or a computer bus or via any other suitable electrical connection to exchange electrical signals. The camera <NUM> may be coupled to the computer system <NUM>, for example via an external interface, or may be provided as parts of the computer system (in other words: internal to the computer system, for example coupled via the electrical connection <NUM>).

The terms "coupling" or "connection" are intended to include a direct "coupling" (for example via a physical link) or direct "connection" as well as an indirect "coupling" or indirect "connection" (for example via a logical link), respectively.

Claim 1:
Computer implemented method for filtering portions of an image,
the method comprising the following steps carried out by computer hardware components:
- dividing (<NUM>) the image into a plurality of segments, each segments comprising a plurality of pixels;
- for each of the segments, determining (<NUM>) at least one of a mean, a standard deviation, and a kurtosis of the plurality of pixels of the respective segment;
- clustering (<NUM>) the plurality of segments into a plurality of clusters based on the at least one of the mean, the standard deviation, and the kurtosis of the plurality of pixels of the respective segment;
- for each of the clusters, determining (<NUM>) the respective cluster as belonging to a background based on a size of the respective cluster; and
- determining (<NUM>) a filtered image based on the background.