Patent Description:
In general terms, object detection is a computer technology related to computer vision and image processing that deals with detecting instances of semantic objects of a certain class (such as human beings, animals, vehicles, etc.). Some object detection algorithms produce key points, or other types of object position indicators. Such points, or indicators, generally represent points of interest for the target object as tracked from one image frame to the next.

There are some scenarios, or environments, where object detection as referred to above becomes challenging. Taking a city environment as an example, this is a type of environment having a large amount of reflective surfaces. The reflective surfaces might be mirrors, but also sheets made of glass, or other types of reflective material. These surfaces generally cause problems for object detection algorithms. In some cases, both a detected object and its reflection will be counted. This could be an issue in applications where the number of detected objects is to be counted, causing some detected objects to be counted twice. For other cases, the reflection might be missed completely. This could be an issue in applications where any detected objects are to be masked, leaving the reflection unmasked.

In <CIT> is disclosed an information processing device that includes a specification circuit and a count circuit. The specification circuit specifies, based on a similarity of speed depending on a change in positions of a plurality of movable objects in an image, two or more movable objects corresponding to a same real movable object in the movable objects. The count circuit counts the number of real movable objects corresponding to the movable objects based on the specification result of the specification circuit. In <CIT> and <CIT> further background information can be found.

However, in practice, a reflection is not perfectly equal to the tracked object it is reflecting. A reflection might appear smaller or larger than the tracked object. Furthermore, reflections made in semitransparent surfaces make the reflection have less details in the image frames than the tracked object. Additionally, the reflection might only include parts of the tracked object. This could cause the specification circuit in the information processing device in <CIT> to make erroneous decisions as to whether two or more movable objects correspond to the same real movable object in the movable objects.

An object of embodiments herein is to address the above issues and to provide improved detection of objects in reflective surfaces.

According to a first aspect there is therefore presented a method for detecting a reflection of an object in a sequence of image frames. The method is defined in the appended set of claims.

According to a second aspect there is presented a controller for detecting a reflection of an object in a sequence of image frames. The controller is defined in the appended set of claims.

According to a third aspect there is presented a video surveillance system. The video surveillance system comprises a controller according the second aspect and a camera for capturing the sequence of image frames.

According to a fourth aspect there is presented a computer program for detecting a reflection of an object in a sequence of image frames, the computer program comprising computer program code which, when run on a controller, causes the controller to perform a method according to the first aspect.

According to a fifth aspect there is presented a computer program product comprising a computer program according to the fourth aspect and a computer readable storage medium on which the computer program is stored. The computer readable storage medium could be a non-transitory computer readable storage medium.

Advantageously, these aspects provide computationally efficient and accurate detection of objects in scenarios, or environments, having reflective surfaces.

<FIG> is a schematic diagram illustrating a video surveillance system <NUM> where embodiments presented herein can be applied. A camera <NUM> is configured to capture image frames, within a Field-of-View (FoV) <NUM>, of a scene <NUM>. In the illustrative example of <FIG> the scene <NUM> comprises an object <NUM>-<NUM> in the form of a human being. The camera <NUM> comprises, is collocated with, is integrated with, or at least operatively connected to, a controller <NUM>. It is assumed that the object <NUM>-<NUM> represents a target object that is to be tracked from image frame to image frame, as captured by the camera <NUM> and as analyzed by the controller <NUM>. For this purpose, the controller <NUM> might implement an object detection algorithm.

The embodiments disclosed herein relate to techniques for detecting a reflection of an object <NUM>-<NUM> in a sequence of image frames. In order to obtain such techniques there is provided a controller <NUM>, a method performed by the controller <NUM>, a computer program product comprising code, for example in the form of a computer program, that when run on a controller <NUM>, causes the controller <NUM> to perform the method.

<FIG> schematically illustrates an image frame <NUM>. The image frame <NUM> depicts an object <NUM>-<NUM> in the form of a human being, as in the scene <NUM>. The object <NUM>-<NUM> has a location marked at reference numeral <NUM>-<NUM> in the image frame <NUM>. In the image frame <NUM> is further depicted a reflection surface <NUM>. The reflection surface <NUM> has a location marked at reference numeral <NUM> in the image frame <NUM>. The reflection surface <NUM> causes also a further object <NUM>-<NUM> to be visible. Object <NUM>-<NUM> is a reflection of object <NUM>-<NUM>, as caused by object <NUM>-<NUM> being reflected by the reflection surface <NUM>. The object <NUM>-<NUM> has a location marked at reference numeral <NUM>-<NUM> in the image frame <NUM>.

The inventors have realized that what object <NUM>-<NUM> and object <NUM>-<NUM> will have in common are their object position indicators. Reference is here made to <FIG> which schematically illustrate an object <NUM>-<NUM> in the form of a human being, as in the scene <NUM>. In <FIG> are shown different types of object position indicators. In <FIG> the object <NUM>-<NUM> is enclosed by a bounding box <NUM>. The bounding box <NUM> is defined by two corner points 320a, 320b and one center point <NUM>. The center point <NUM> could represent the location <NUM>-<NUM> of the object <NUM>-<NUM>. The corner points 320a, 320b and the center point <NUM> could, for instance, be CenterNet keypoints. The bounding box could, for instance, be a CenterNet bounding box. In this respect, the object position indicators might be defined by two or more bounding boxes. That is, two or more bounding boxes might be used to enclose the detected object <NUM>-<NUM>. That is, a first bounding box might enclose a first part (such as the head) of the detected object <NUM>-<NUM>, a second bounding box might enclose a second part (such as the torso) of the detected object <NUM>-<NUM>, and so on. In <FIG> the object <NUM>-<NUM> is overlayed by object position indicators <NUM>-<NUM> to 417b-<NUM> as listed in Table <NUM>. These object position indicators could be regarded as a simplified version of COCO key points or MediaPipe Pose key points.

The object <NUM>-<NUM> being the reflection will have an identical setup of object position indicators but reflected. By analyzing the object position indicators, it is possible to determine object-reflection-pairs, at object level i.e., one tracked object <NUM>-<NUM> and its reflected counterpart object <NUM>-<NUM> or even at object position indicator level. Reference is here made to <FIG> which schematically illustrates object position indicators <NUM>-<NUM> to 417b-<NUM> as listed in Table <NUM> of the object <NUM>-<NUM> (not shown). <FIG> also indicates the position <NUM>-<NUM> of the object <NUM>-<NUM>. <FIG> further illustrates object position indicators <NUM>-<NUM> to 417b-<NUM> of the object <NUM>-<NUM> (not shown) being a reflection of the object <NUM>-<NUM> in the reflection surface <NUM> at location <NUM>. The object position indicators <NUM>-<NUM> to 417b-<NUM> correspond to the object position indicators listed in Table <NUM>. That is, object position indicator <NUM>-<NUM> represents the head of the object <NUM>-<NUM>, and object position indicator 417b-<NUM> represents the left foot of the object <NUM>-<NUM>.

<FIG> is a flowchart illustrating embodiments of methods for detecting a reflection of an object <NUM>-<NUM> in a sequence of image frames <NUM>, <NUM>, <NUM>. The methods are performed by the controller <NUM>. The methods are advantageously provided as computer programs <NUM>.

S102: The controller <NUM> detects objects <NUM>-<NUM>, <NUM>-<NUM> of a given type in the sequence of image frames <NUM>, <NUM>, <NUM>. The controller <NUM> determines a detection score for each detected object <NUM>-<NUM>, <NUM>-<NUM>.

In some examples, a detection score is also determined for each object position indicator identified for the detected object <NUM>-<NUM>, <NUM>-<NUM> in addition to the detection score that is determined for each detected object <NUM>-<NUM>, <NUM>-<NUM> as a whole. The detection scores of the individual object position indicators might only become relevant once the detection score of the detected object <NUM>-<NUM>, <NUM>-<NUM> is above some threshold value. Conversely, in some aspects the detection score for a given detected object <NUM>-<NUM>, <NUM>-<NUM> is a function of the detection scores for all unfiltered object position indicators identified for the given detected object <NUM>-<NUM>, <NUM>-<NUM>.

S104: The controller <NUM> determines, per each detected object <NUM>-<NUM>, <NUM>-<NUM>, distance ratios between unfiltered object position indicators identified for the detected object <NUM>-<NUM>, <NUM>-<NUM>. The unfiltered object position indicators are identified in a current image frame <NUM>, <NUM>, <NUM> in the sequence of image frames <NUM>, <NUM>, <NUM>. The distance ratios for the detected object <NUM>-<NUM>, <NUM>-<NUM> define a normalized size of the detected object <NUM>-<NUM>, <NUM>-<NUM>.

Since the size of the detected object is determined per the detected object as a whole, the size is considered to be the normalized size of the detected object.

S106: The controller <NUM> determines, per each detected object <NUM>-<NUM>, <NUM>-<NUM>, a displacement factor between a current location <NUM>-<NUM>, <NUM>-<NUM> of the detected object <NUM>-<NUM>, <NUM>-<NUM> in the current image frame <NUM>, <NUM>, <NUM> and a previous location of the detected object <NUM>-<NUM>, <NUM>-<NUM> in a previous image frame in the sequence of image frames <NUM>, <NUM>, <NUM>. The displacement factor for the detected object <NUM>-<NUM>, <NUM>-<NUM> defines a normalized movement for the detected object <NUM>-<NUM>, <NUM>-<NUM>.

Hence, historical data, in terms of a previous image frame in the sequence of image frames <NUM>, <NUM>, <NUM>, is used for the controller <NUM> to obtain information of the location of the detected object in the previous image frame. Based on how much the detected object has moved from the previous image frame to the current image frame, a measure of the movement of the detected object can be obtained. Since the movement for the detected object is determined per the detected object as a whole, the movement is considered to be the normalized movement for the detected object.

The normalized sizes and the normalized movements of two of the detected objects <NUM>-<NUM>, <NUM>-<NUM> can then be compared to each other. That is, the normalized size of one of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> is compared to the normalized size of the other of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> and the normalized movement of one of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> is compared to the normalized movement of the other of the two detected objects <NUM>-<NUM>, <NUM>-<NUM>. Further in this respect, in general terms, the normalized sizes and the normalized movements imply that certain given measures (the size for the normalized sizes and the movement for the normalized movements) are scaled, or otherwise recalculated, to a common scale.

Assuming that the object <NUM>-<NUM> and the reflection of the object are seen at mutually different distances to the camera <NUM>, the effect will be that the object <NUM>-<NUM> and the reflection of the object will appear to be of mutually different sizes; the one being located closest to the camera will appear to be larger, and vice versa. By using normalized sizes, the object <NUM>-<NUM> and the reflection of the object will be scaled, or otherwise recalculated, to a common size scale where this effect will be accounted for such that the object <NUM>-<NUM> and the reflection of the object will appear to be of one and the same size. Likewise, under the same assumption, the effect will also be that the object <NUM>-<NUM> and the reflection of the object will appear to move at mutually different speeds; the one being located closest to the camera will appear to move faster, and vice versa. By using normalized movements, the object <NUM>-<NUM> and the reflection of the object will be scaled, or otherwise recalculated, to a common movement scale where this effect will be accounted for such that the object <NUM>-<NUM> and the reflection of the object will appear to move at one and the same speed.

If the normalized sizes and the normalized movements are similar for the two detected objects <NUM>-<NUM>, <NUM>-<NUM>, then it can be concluded that these two detected objects <NUM>-<NUM>, <NUM>-<NUM> have similar size and similar movement. This is an indication that one of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> is a reflection of the other of the two detected objects <NUM>-<NUM>, <NUM>-<NUM>.

S110: The controller <NUM> determines, in response to finding a match between the normalized sizes and the normalized movements of two of the detected objects <NUM>-<NUM>, <NUM>-<NUM>, that the one of these two detected objects <NUM>-<NUM>, <NUM>-<NUM> which has a lower detection score is a reflection of the other of these two detected objects <NUM>-<NUM>, <NUM>-<NUM>.

Hence, by considering the detection scores of the two detected objects <NUM>-<NUM>, <NUM>-<NUM>, the detected object <NUM>-<NUM> with a comparatively low detection score is classified as a reflection of the detected object <NUM>-<NUM> with a comparatively high detection score. Embodiments relating to further details of detecting a reflection of an object <NUM>-<NUM> in a sequence of image frames <NUM>, <NUM>, <NUM> as performed by the controller <NUM> will now be disclosed.

As disclosed above, the controller <NUM> detects objects <NUM>-<NUM>, <NUM>-<NUM> of a given type in the sequence of image frames <NUM>, <NUM>, <NUM>. The given type is a human being, an animal, or a vehicle, such as a car or a truck. The controller <NUM> could therefore have been trained, and thereby configured, to detect objects <NUM>-<NUM>, <NUM>-<NUM> of the given type.

As disclosed above, the controller <NUM> determines, per each detected object <NUM>-<NUM>, <NUM>-<NUM>, distance ratios between unfiltered object position indicators identified for the detected object <NUM>-<NUM>, <NUM>-<NUM>. In general terms, the unfiltered object position indicators are object position indicators that have been determined without any objects that have been detected with low detection scores first having been filtered out. That is, unfiltered object position indicators refers to the object position indicators before thresholding is performed to mitigate the false positives in the scene. Since reflections generally have a lower detection score than their non-reflected counterpart, unfiltered object position indicators are used so that the detected object <NUM>-<NUM> being the reflection is not missed. That is, some of the unfiltered object position indicators might belong to objects that would otherwise be filtered out, or discarded, due to producing a low detection score. Hence, according to the invention, the unfiltered object position indicators are determined for all detected objects <NUM>-<NUM>, <NUM>-<NUM> without any of the detected objects <NUM>-<NUM>, <NUM>-<NUM> having been filtered out (due to their detection score being lower than a detection threshold). Further in this respect, there could be different types of object position indicators.

In some non-limiting examples, each of the object position indicators is a Common Objects in Context, COCO, key point <NUM>-<NUM>:417b-<NUM>, <NUM>-<NUM>:417b-<NUM>, or a MediaPipe Pose key point. In this respect, the object position indicators need not to be defined by all COCO key points or MediaPipe Pose key points, but rather a subset of these key points could be used as object position indicators. This is illustrated in <FIG> as referred to above.

In other non-limiting examples, the object position indicators are corner points <NUM>, 320b and center points <NUM> of bounding boxes <NUM>, such as CenterNet bounding boxes. This is illustrated in <FIG> as referred to above.

As disclosed below, that one of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> is a reflection of the other of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> is determined in response to finding a match between the normalized sizes and/or the normalized movements of these two of the detected objects <NUM>-<NUM>, <NUM>-<NUM>. Therefore, in some embodiments, the controller <NUM> is configured to perform (optional) step S108.

S108: The controller <NUM> finds the match between the normalized sizes and/or the normalized movements of two of the detected objects <NUM>-<NUM>, <NUM>-<NUM>.

There could be different ways for the controller <NUM> to find the match between the normalized sizes and/or the normalized movements of two of the detected objects <NUM>-<NUM>, <NUM>-<NUM>. In some aspects, this match-finding is performed by comparing the normalized size and/or the normalized movement of a first detected object <NUM>-<NUM> to the normalized size and/or the normalized movement of a second detected object <NUM>-<NUM>. In particular, in some embodiments, the controller <NUM> is configured to perform (optional) step S108a and/or step S108b as part of step S108.

S108a: The controller <NUM> confirms that a size difference between the normalized size of one of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> and the normalized size of the other of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> is smaller than a size threshold.

Hence, the smaller the size difference between the normalized size of one of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> and the normalized size of the other of the two detected objects <NUM>-<NUM>, <NUM>-<NUM>, the higher the chance that there is a match.

S108b: The controller <NUM> confirms that a movement difference between the normalized movement of one of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> and the normalized movement of the other of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> is smaller than a movement threshold.

Hence, the smaller the movement difference between the normalized movement of one of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> and the normalized movement of the other of the two detected objects <NUM>-<NUM>, <NUM>-<NUM>, the higher the chance that there is a match.

In some aspects, the matching is performed on an object position indicator level. That is, instead of just comparing the normalized size and/or the normalized movement of a first detected object <NUM>-<NUM> to the normalized size and/or the normalized movement of a second detected object <NUM>-<NUM>, a comparison (with respect to size and/or movement) is made between pairs of object position indicators in one of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> and pairs of object position indicators in the other of the two detected objects <NUM>-<NUM>, <NUM>-<NUM>.

That is, assume for illustrative purposes that the two detected objects <NUM>-<NUM>, <NUM>-<NUM> represent a human being and the knees of both legs are detectable and are represented by object position indicators in both the two detected objects <NUM>-<NUM>, <NUM>-<NUM>. Then a first comparison can be made between the normalized distance between the knees according to the object position indicators in one of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> and the normalized distance between the knees according to the object position indicators in the other of the two detected objects <NUM>-<NUM>, <NUM>-<NUM>. Further, a second comparison can be made between the normalized movement of the knees, as given by the object position indicators, in one of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> and the normalized movement of the knees, as given by the object position indicators, of the other of the two detected objects <NUM>-<NUM>, <NUM>-<NUM>. Further such comparisons with respect to size and/or movement can be made for further parts of the detected objects to accumulate a matching score for the two detected objects <NUM>-<NUM>, <NUM>-<NUM>.

In some aspects, the controller <NUM> performs some action upon determination that one of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> is a reflection of the other of the two detected objects <NUM>-<NUM>, <NUM>-<NUM>. Hence, in some embodiments, the controller <NUM> is configured to perform (optional) step S112.

S112: The controller <NUM> performs an action with respect to the detected object <NUM>-<NUM> being the reflection of the other detected object <NUM>-<NUM>.

There could be different types of actions performed by the controller <NUM> in step S112. In some non-limiting examples, the action involves applying a mask <NUM>-<NUM> to the detected object <NUM>-<NUM> being the reflection of the other detected object <NUM>-<NUM>. Intermediate reference is here made to <FIG> which schematically illustrates an image frame <NUM>. The image frame <NUM> depicts how a first mask <NUM>-<NUM> has been placed where object <NUM>-<NUM> appeared in <FIG> and a second mask <NUM>-<NUM> has been placed where object <NUM>-<NUM> appeared in <FIG>. In some non-limiting examples, the action involves filtering out the detected object <NUM>-<NUM> being the reflection of the other detected object <NUM>-<NUM>, lowering the detection score threshold for the detected object <NUM>-<NUM> being the reflection of the other detected object <NUM>-<NUM>, etc. Intermediate reference is here made to <FIG> which schematically illustrates an image frame <NUM>. The image frame <NUM> depicts an object <NUM>-<NUM> as in <FIG>, but where object <NUM>-<NUM> has been filtered out to not appear in image frame <NUM>. In this respect, although object <NUM>-<NUM> representing the reflection is illustrated as being filtered out from the image frame, it is noted that the filtering out generally refers to filtering out the detected object <NUM>-<NUM> from some counting, or other type of calculation, such that object <NUM>-<NUM> is not counted twice.

In some aspects, as shown in <FIG>, the reflection is caused by a reflection surface <NUM> being placed between the two detected objects <NUM>-<NUM>, <NUM>-<NUM>. In general terms, the reflection surface <NUM> is placed at a midpoint between the locations <NUM>-<NUM>, <NUM>-<NUM> of the two detected objects <NUM>-<NUM>, <NUM>-<NUM>. Therefore, knowledge of the locations <NUM>-<NUM>, <NUM>-<NUM> of the two detected objects <NUM>-<NUM>, <NUM>-<NUM> is used to determine the location <NUM> of the reflection surface <NUM>. Hence, in some embodiments, the controller <NUM> is configured to perform (optional) step S114.

S114: The controller <NUM> determines a location <NUM> of a reflection surface <NUM> causing the one of the detected objects <NUM>-<NUM>, <NUM>-<NUM> to be the reflection of the other detected object <NUM>-<NUM>, <NUM>-<NUM>. The location <NUM> of the reflection surface <NUM> is determined as a midpoint between the locations <NUM>-<NUM>, <NUM>-<NUM> of the two detected objects <NUM>-<NUM>, <NUM>-<NUM>.

In some embodiments, knowledge of the location <NUM> of the reflection surface <NUM> is utilized for detection of further objects in the sequence of image frames <NUM>, <NUM>, <NUM>. Since the location <NUM> of the reflection surface <NUM> can be used as a reference point, or reference line, or even reference surface, knowledge of the location <NUM> could improve the chance of detecting reflected objects in challenging lighting conditions in future image frames of the same scene, or at least scenes where the reflection surface <NUM> is still present in the scene.

It is noted that by the matching referring to the normalized sizes of the objects <NUM>-<NUM>, <NUM>-<NUM>, no consideration needs to be made with respect to that the object position indicators <NUM>-<NUM> to 417b-<NUM> of the object <NUM>-<NUM> are mirrored compared to the object position indicators <NUM>-<NUM> to 417b-<NUM> of the object <NUM>-<NUM>. For the same reason, by the matching referring to the normalized movements of the objects <NUM>-<NUM>, <NUM>-<NUM>, the normalized movement for object <NUM>-<NUM> will be the same as the normalized movements for object <NUM>-<NUM> except for a possible change of sign, due to the placement of the reflection surface <NUM> with respect to the detected objects <NUM>-<NUM>, <NUM>-<NUM>. For example, with the placement of the reflection surface <NUM> as in <FIG>, if object <NUM>-<NUM> appears to move towards the left in <FIG>, then object <NUM>-<NUM> appears to move towards the right in <FIG>. To resolve this potential issue, the normalized movement might be represented by only its magnitude, without considering its direction. Further, for any matching performed on object position indicator level, then the reflection should be taken into considerations when the matching involves any object position indicators for which there is both a right object position indicator and a left object position indicator per each object, such as object position indicators 412a-<NUM>, 412b-<NUM>, etc..

The thus far disclosed methods and controller <NUM> might be used as part of techniques for improving detection of human beings and other objects <NUM>-<NUM> in scenes <NUM> with glass surfaces. Glass surfaces may be both reflective and transmissive. A reflected image may be dimmer than the image of the actual person or object. The same is true for a human being, or other object <NUM>-<NUM>, visible through the glass, particularly under some light conditions. As already disclosed, if the purpose of detecting the object <NUM>-<NUM> is to mask the object <NUM>-<NUM>, then there is a risk that a mirror image of an object <NUM>-<NUM> remains unmasked and possibly identifiable if thresholds for detection are too strict. Conversely, if the purpose of detecting the object <NUM>-<NUM> is for counting, then there is a risk of double counting the object <NUM>-<NUM> if thresholds for detection are too generous.

Therefore, in some aspects, areas are identified where reflected images and transmissive images are likely to occur. It may be easier to identify such areas under certain lighting conditions and knowledge gained during beneficial conditions may be used during more difficult conditions. If a surface can be identified where reflections <NUM>-<NUM> of objects <NUM>-<NUM> often are found, but equally often reflections are not found (e.g., the keypoints do not have a match), it is likely that this surface is both reflective and transmissive.

Lighting conditions might impact how reflected and/or transmitted an object <NUM>-<NUM>, <NUM>-<NUM> will be. For example, it is generally comparatively easier to study surfaces with high reflection or transmission, as this makes the detection more distinct, and therefore more reliable. On the other hand, it is generally difficult to detect objects under dim lighting conditions. Information of the lighting conditions might therefore be used when determining whether an object is a reflection or not. For example, thresholds may be lowered when dim lighting conditions are indicated. This may also help in tuning the keypoint reflection algorithm. For example, if it is found that in a given area of the image it is more likely to find reflections rather than transmissions, then a more aggressive search for keypoint pairs can be performed in this given area of the image. Further, bright lighting conditions generally create more reflected objects than transmissive objects. This knowledge may also be used for aiding the algorithm. Thus, if the lighting conditions indicate a high likelihood of reflections, then a more aggressive search for keypoint pairs may be performed.

A possible approach will now be briefly described. Reflections can be found using the methods as described above. The number of recorded reflections in a given part of the scene <NUM> is saved over time. This could be regarded as generating a heatmap. If there are many recorded reflections, but also a lot of true targets for a given surface, this is marked in the heatmap. For a privacy masking application, a static mask may be applied on this surface to ensure that reflections are masked, even in case they fall below a current detection threshold. Alternatively, the masking threshold required to mask an object <NUM>-<NUM> may be lowered in this area. For each image frame <NUM>, it may be possible to determine if a detected object <NUM>-<NUM>, <NUM>-<NUM> is a reflection or not by finding reflection pairs in the scene <NUM>. Thus, by building data over time using the probability of an object <NUM>-<NUM> being a reflection or not, it may be possible to in the future predict the probability of a new object located in the same area of the image frame being a reflection or not.

<FIG> schematically illustrates, in terms of a number of functional units, the components of a controller <NUM> according to an embodiment. Processing circuitry <NUM> is provided using any combination of one or more of a suitable central processing unit (CPU), multiprocessor, microcontroller, digital signal processor (DSP), etc., capable of executing software instructions stored in a computer program product <NUM> (as in <FIG>), e.g. in the form of a storage medium <NUM>. The processing circuitry <NUM> may further be provided as at least one application specific integrated circuit (ASIC), or field programmable gate array (FPGA).

Particularly, the processing circuitry <NUM> is configured to cause the controller <NUM> to perform a set of operations, or steps, as disclosed above. For example, the storage medium <NUM> may store the set of operations, and the processing circuitry <NUM> may be configured to retrieve the set of operations from the storage medium <NUM> to cause the controller <NUM> to perform the set of operations.

Thus the processing circuitry <NUM> is thereby arranged to execute methods as herein disclosed. The controller <NUM> may further comprise a communications interface <NUM> at least configured for communications with the camera <NUM>, potentially with other functions, nodes, entities and/or devices, such as functions, nodes, entities and/or devices of a video surveillance system. As such the communications interface <NUM> may comprise one or more transmitters and receivers, comprising analogue and digital components.

The processing circuitry <NUM> controls the general operation of the controller <NUM>, e.g., by sending data and control signals to the communications interface <NUM> and the storage medium <NUM>, by receiving data and reports from the communications interface <NUM>, and by retrieving data and instructions from the storage medium <NUM>. Other components, as well as the related functionality, of the controller <NUM> are omitted in order not to obscure the concepts presented herein.

The controller <NUM> may be provided as a standalone device or as a part of at least one further device. For example, the controller <NUM> and the camera <NUM> might be part of a video surveillance system <NUM>. Optionally, the controller <NUM> may be integrated in the camera <NUM>. A first portion of the instructions performed by the controller <NUM> may be executed in a first device, and a second portion of the of the instructions performed by the controller <NUM> may be executed in a second device; the herein disclosed embodiments are not limited to any particular number of devices on which the instructions performed by the controller <NUM> may be executed. Hence, the methods according to the herein disclosed embodiments are suitable to be performed by a controller <NUM> residing in a cloud computational environment. Therefore, although a single processing circuitry <NUM> is illustrated in <FIG> the processing circuitry <NUM> may be distributed among a plurality of devices, or nodes. The same applies to the computer program <NUM> of <FIG>.

Claim 1:
A method for detecting a reflection of an object (<NUM>-<NUM>) in a sequence of image frames (<NUM>, <NUM>, <NUM>), the method being performed by a controller (<NUM>), the method comprising:
detecting (S102) objects (<NUM>-<NUM>, <NUM>-<NUM>) of a given type in the sequence of image frames (<NUM>, <NUM>, <NUM>), wherein the given type is a human being, an animal, or a vehicle, such as a car or a truck, and determining a detection score for each detected object (<NUM>-<NUM>, <NUM>-<NUM>);
determining (S104), per each detected object (<NUM>-<NUM>, <NUM>-<NUM>), distance ratios between unfiltered object position indicators identified for the detected object (<NUM>-<NUM>, <NUM>-<NUM>), the unfiltered object position indicators being identified in a current image frame (<NUM>, <NUM>, <NUM>) in the sequence of image frames (<NUM>, <NUM>, <NUM>), wherein object position indicators represent points of interest for the target object as tracked from one image frame to the next, the distance ratios for the detected object (<NUM>-<NUM>, <NUM>-<NUM>) defining a normalized size of the detected object (<NUM>-<NUM>, <NUM>-<NUM>), wherein the unfiltered object position indicators are determined for all detected objects (<NUM>-<NUM>, <NUM>-<NUM>) without any of the detected objects (<NUM>-<NUM>, <NUM>-<NUM>) having been filtered out, due to their detection score being lower than a detection threshold;
determining (S106), per each detected object (<NUM>-<NUM>, <NUM>-<NUM>), a displacement factor between a current location (<NUM>-<NUM>, <NUM>-<NUM>) of the detected object (<NUM>-<NUM>, <NUM>-<NUM>) in the current image frame (<NUM>, <NUM>, <NUM>) and a previous location of the detected object (<NUM>-<NUM>, <NUM>-<NUM>) in a previous image frame in the sequence of image frames (<NUM>, <NUM>, <NUM>), the displacement factor for the detected object (<NUM>-<NUM>, <NUM>-<NUM>) defining a normalized movement for the detected object (<NUM>-<NUM>, <NUM>-<NUM>), wherein the normalized movement refers to movement in a common scale with respect to camera distance; and
determining (S110), in response to finding a match between the normalized sizes and the normalized movements of two of the detected objects (<NUM>-<NUM>, <NUM>-<NUM>), that the one of these two detected objects (<NUM>-<NUM>, <NUM>-<NUM>) which has a lower detection score is a reflection of the other of these two detected objects (<NUM>-<NUM>, <NUM>-<NUM>).