Patent Description:
In general terms, surveillance cameras are video cameras used for the purpose of observing an area. Surveillance cameras are often connected to a recording device or a network, such as an internet protocol (IP) network. In some examples, images recorded by the surveillance cameras are monitored in real-time by a security guard or law enforcement officer. Cameras and other type of recording equipment used to be relatively expensive and required human personnel to monitor camera footage, but automated software has been developed for analysis of footage in terms of digital images captured by digital image video cameras. Such automated software can be configured for digital image analysis as well as organization of the thus captured digital video footage into a searchable database.

In some surveillance scenarios, there could be a need to capture a scene having a Field of View (FoV) including one or more regions for which privacy should be preserved. The image content of such one or more regions should thus be excluded from any recording in the video surveillance system. In some surveillance scenarios, where a surveillance zone is defined there could be a need to capture a scene having a FoV that is wider than the surveillance zone. Since only image content from inside the surveillance zone need to be analyzed, any image content outside the surveillance zone should be excluded from any analysis in the video surveillance system (although the image content outside the surveillance zone does not need to be excluded from any recording in the video surveillance system). In respect to both these scenarios, a region of interest (ROI) can be defined as a portion of an image that is to be subjected to filtering or other type of operation. An ROI can thus be defined such that the one or more regions for which privacy should be preserved is filtered out. An ROI can be defined by a binary mask. The binary mask is of the same size as the image to be processed. In the binary mask, pixels that define the ROI are set to <NUM> whereas all other pixels are set to <NUM> (or vice versa, depending on the purpose of the binary mask; to include or exclude the image content inside the ROI).

In other surveillance scenarios, a virtual line extending between two end-points, and defining a virtual trip wire, could by the video surveillance system be added in the scene for crossline detection; an alarm, or other type of indication, is to be issued whenever an object (such as a human being, vehicle, or other type of object) crosses the virtual line in the scene.

One way to define the corner points of an ROI and/or the end-points of the virtual line is to use annotated points. Each annotated point has a coordinate which commonly is manually or semi-automatically set. However, once set, the coordinates of the annotated points might suffer from camera drifting due to vibration, camera position drift, FoV orientation shift, temperature changes, mechanical aging, and so on. This might cause the corner points of the ROI and/or the end-points of the virtual line to change, thereby affecting the location of the privacy mask, which part of the surveillance zone that is actually under surveillance, and/or the location of the virtual line. Further, when the camera is replaced, the annotated points need to be manually relabelled.

Hence, there is a need for an improved handling of annotated points when a camera is subjected to camera movement (as, for example, caused by any of: camera drifting due to vibration, camera position drift, FoV orientation shift, temperature changes, mechanical aging, camera replacement).

<NPL> relates to a video annotation tool using vision-based Augmented Reality (AR) technology. AR technology and computer vision methods are applied for making videos with 3D annotations.

<NPL> relates to a method for points tracking or transferring based on the Kanade-Lucas-Tomasi (KLT) feature tracker and the projective reconstruction technique. The points to be tracked include the lost natural features, and any points that are specified by the users. The proposed method is useful for AR applications, including scene annotation, registration, etc..

<CIT> relates to systems, devices, and techniques for applying annotations from a first image to a second image based on a set of keypoints and an image patch.

An object of embodiments herein is to address the above issues by providing a method, an image processing device, a computer program, and a computer program product for updating a coordinate of an annotated point in a digital image due to camera movement (where, as above, the camera movement is caused by any of: camera drifting due to vibration, camera position drift, FoV orientation shift, temperature changes, mechanical aging, camera replacement).

According to a first aspect there is presented a method as defined in claim <NUM> for updating a coordinate of an annotated point in a digital image due to camera movement. The method is performed by an image processing device. The method comprises obtaining a current digital image of a scene. The current digital image has been captured by a camera subsequent to movement of the camera relative the scene. The current digital image is associated with at least one annotated point. Each at least one annotated point has a respective coordinate in the current digital image. The method comprises identifying an amount of the movement by comparing position indicative information in the current digital image to position indicative information in a previous digital image of the scene. The previous digital image has been captured prior to movement of the camera. The method comprises updating the coordinate of each at least one annotated point in accordance with the identified amount of movement and a camera homography.

According to a second aspect there is presented an image processing device as defined in claim <NUM> for updating a coordinate of an annotated point in a digital image due to camera movement. The image processing device comprises processing circuitry. The processing circuitry is configured to cause the image processing device to obtain a current digital image of a scene. The current digital image has been captured by a camera subsequent to movement of the camera relative the scene. The current digital image is associated with at least one annotated point. Each at least one annotated point has a respective coordinate in the current digital image. The processing circuitry is configured to cause the image processing device to identify an amount of the movement by comparing position indicative information in the current digital image to position indicative information in a previous digital image of the scene. The previous digital image has been captured prior to movement of the camera. The processing circuitry is configured to cause the image processing device to update the coordinate of each at least one annotated point in accordance with the identified amount of movement and a camera homography.

There presented a video surveillance system. The video surveillance system comprises a camera and an image processing device according to the second aspect.

According to a third aspect there is presented a computer program as defined in claim <NUM> for updating a coordinate of an annotated point in a digital image due to camera movement, the computer program comprising computer program code which, when run on an image processing device, causes the image processing device to perform a method according to the first aspect.

Advantageously, these aspects resolve the issues noted above with respect to camera movement.

Advantageously, these aspects provide accurate updating of the coordinate of the annotated point in the digital image due to camera movement.

Advantageously, these aspects enable automatic re-location, or recalibration, of annotated points when they shift due to camera movement.

The embodiments are provided by way of example so that this disclosure will be thorough and complete, and will fully convey the scope of the inventive concept as defined by the appended claims to those skilled in the art.

<FIG> is a schematic diagram illustrating a scenario where embodiments presented herein can be applied. The schematic diagram of <FIG> represents a surveillance scenario. A camera <NUM> is configured to capture digital images, within a FoV <NUM>, of a scene <NUM>. In the illustrative example of <FIG> the scene <NUM> comprises a vehicle <NUM> and a person <NUM>. The camera <NUM> comprises an image processing device <NUM>. The camera <NUM> and the image processing device <NUM> might be part of a video surveillance system. Such a video surveillance system could be applied to any of the above surveillance scenarios.

As noted above there is a need for an improved handling of annotated points when a camera <NUM> is subjected to camera movement (as, for example, caused by any of: camera drifting due to vibration, camera position drift, FoV orientation shift, temperature changes, mechanical aging, camera replacement).

The embodiments disclosed herein therefore relate to mechanisms for updating a coordinate of an annotated point in a digital image due to camera movement. In order to obtain such mechanisms there is provided an image processing device <NUM>, a method performed by the image processing device <NUM>, and a computer program product comprising code, for example in the form of a computer program, that when run on an image processing device <NUM>, causes the image processing device <NUM> to perform the method.

The embodiments disclosed herein enable feature-linked annotated points (of a previous digital image) to automatically calibrate the annotated points (of the current digital image) to solve the above issues relating to camera movement caused by any of camera drifting due to vibration, camera position drift, FoV orientation shift, temperature changes, mechanical aging, camera replacement. The embodiments disclosed herein are based on detecting movement of the camera <NUM> and updating at least one coordinate of an annotated points by comparing position indicative information of a current digital image to that of a previous digital image of the same scene <NUM>. This will now be illustrated with reference to <FIG> schematically illustrates annotated points a<NUM>, a<NUM> as well as position indicative information in terms of key points k<NUM>, k<NUM>, k<NUM>, k<NUM>, k<NUM>, k<NUM> in a previous digital image 200a. <FIG> schematically illustrates the same annotated points <MAT> and key points <MAT> as in the previous digital image 200a but in a current digital image 200b of the same scene as the previous digital image 200a. In <FIG> is also marked at C the image center and a radius r extending from the image center C. Possible usage of the parameters C and r will be disclosed below. It is assumed that camera movement has occurred sometime in between capturing of the previous digital image 200a and capturing of the current digital image 200b. Therefore, as can be seen by comparing <FIG>, the camera movement has caused the coordinates of the annotated points a<NUM>, a<NUM> (as well as the coordinates of the key points k<NUM>:k<NUM>) to be shifted.

<FIG> is a flowchart illustrating embodiments of methods for updating a coordinate of an annotated point a<NUM>, a<NUM> in a digital image 200b due to camera movement. The camera movement can be due to either one and the same camera <NUM> having been moved so that the FoV <NUM> changes, or that one camera <NUM> is replaced by another camera <NUM> (which, potentially, also causes the FoV <NUM> to change). The methods are performed by the image processing device <NUM>. The methods are advantageously provided as computer programs <NUM>. In particular, the image processing device <NUM> is configured to perform steps S102, S104, S106:
S102: The image processing device <NUM> obtains a current digital image 200b of a scene <NUM>. The current digital image 200b has been captured by the camera <NUM> subsequent to movement of the camera <NUM> relative the scene <NUM>. The current digital image 200b is associated with at least one annotated point <MAT>. Each at least one annotated point <MAT> has a respective coordinate in the current digital image 200b.

S104: The image processing device <NUM> identifies the amount of the movement by comparing position indicative information in the current digital image 200b to position indicative information in a previous digital image 200a of the scene <NUM>. The previous digital image 200a has been captured prior to movement of the camera <NUM> (either by the same camera <NUM> or by another camera <NUM>).

S106: The image processing device <NUM> updates the coordinate of each at least one annotated point (from a<NUM>, a<NUM> to <MAT>) in accordance with the identified amount of movement and a camera homography (of the camera <NUM>).

This method enables automatic re-location, or recalibration, of annotated points when they shift due to camera movement.

Embodiments relating to further details of updating a coordinate of an annotated point <MAT> in a digital image 200b due to camera movement as performed by the image processing device <NUM> will now be disclosed.

There can be different examples of position indicative information. In some aspects, the position indicative information in the current digital image 200b and the previous digital image 200a is defined by a respective set of keypoints k<NUM>:k<NUM>, <MAT>. In particular, in some embodiments, the position indicative information in the current digital image 200b is represented by a first set of keypoints <MAT> extracted from the current digital image 200b, and the position indicative information in the previous digital image 200a is represented by a second set of keypoints k<NUM>:k<NUM> extracted from the previous digital image 200a.

In some aspects, it is the matching between the keypoints <MAT> in the current digital image 200b and the keypoints k<NUM>:k<NUM> in the previous digital image 200a that yields the amount of camera movement. In particular, in some embodiments, the comparing involves location-wise matching of the first set of keypoints <MAT> to the second set of keypoints k<NUM>:k<NUM>. The amount of movement is then identified from how much the coordinates in the current digital image 200b of the first set of keypoints k<NUM>:k<NUM> differ from the coordinates in the previous digital image 200a of the second set of keypoints k<NUM>:k<NUM>.

How discriminative keypoints can be selected based on ranking their similarity will be disclosed next. In some aspects, the matching is performed by means of minimizing the distance between feature vectors in the current digital image 200b and feature vectors in the previous digital image 200a. In particular, in some embodiments, a respective first feature vector is determined for each keypoint k<NUM>:k<NUM> in the first set of keypoints <MAT> and a respective second feature vector is determined for each keypoint k<NUM>:k<NUM> in the second set of keypoints k<NUM>:k<NUM>. The first set of keypoints <MAT> is then matched to the second set of keypoints k<NUM>:k<NUM> by finding a set of pairs of one of the first feature vectors and one of the second feature vectors that yields minimum distance between the first feature vectors and the second feature vectors among all sets of pairs of one of the first feature vectors and one of the second feature vectors.

Properties of the second set of keypoints k<NUM>:k<NUM> will now be disclosed. In some embodiments, the second set of keypoints k<NUM>:k<NUM> is a subset of all keypoints k<NUM>:k<NUM> extractable from the previous digital image 200a. How selection of the most appropriate keypoints can be performed, in terms of how this subset of all keypoints k<NUM>:k<NUM> can be determined, will now be disclosed.

In some aspects, any keypoints k<NUM>:k<NUM> with similar pairs of feature vectors are excluded from being included in the second set of keypoints k<NUM>:k<NUM>. That is, in some embodiments, those of all keypoints k<NUM>:k<NUM> with second feature vectors most similar to a second feature vector of another keypoint k<NUM>:k<NUM> extracted from the previous digital image 200a are excluded from being included in the second set of keypoints k<NUM>:k<NUM>.

In some aspects, location information is used when determining which of all keypoints k<NUM>:k<NUM> to be included in the second set of keypoints k<NUM>:k<NUM>. In particular, in some embodiments, which of all keypoints k<NUM>:k<NUM> to be included in the second set of keypoints k<NUM>:k<NUM> is dependent on their location in the previous digital image 200a. In this respect, as will be disclosed next, the location could be relative to the image center and/or the annotated points.

In some aspects, those of all keypoints k<NUM>:k<NUM> located closest to the image center are candidates to be included in the second set of keypoints k<NUM>:k<NUM>. In particular, in some embodiments, the second set of keypoints k<NUM>:k<NUM> is restricted to only include keypoints k<NUM>:k<NUM> located within a predefined radius, denoted r in <FIG>, from a center, denoted C in <FIG>, of the previous digital image 200a.

In some aspects, those of all keypoints k<NUM>:k<NUM> located closest to the annotated points a<NUM>, a<NUM> are candidates to be included in the in the second set of keypoints k<NUM>:k<NUM>. In particular, in some embodiments, the previous digital image 200a is associated with at least one annotated point a<NUM>, a<NUM>, where each at least one annotated point a<NUM>, a<NUM> has a respective coordinate in the previous digital image 200a, and the second set of keypoints k<NUM>:k<NUM> is restricted to only include keypoints k<NUM>:k<NUM> located within a predefined radius from the coordinate of the at least one annotated point a<NUM>, a<NUM> in the previous digital image 200a.

Thereby, the above embodiments, aspects, and examples enable the most appropriate keypoints to be selected according to the coordinate of the at least one annotated point, and the image centre.

In some aspects, the previous digital image 200a is selected from a set of previous digital images 200a of the scene <NUM>. Examples of how the previous digital image 200a can be selected from this set of previous digital images 200a of the scene <NUM> will now be disclosed.

In some embodiments, each of the previous digital images 200a in the set of previous digital images 200a has its own time stamp, and which previous digital image 200a to select is based on comparing the time stamps to a time stamp of the current digital image 200b. The previous digital image 200a can thus be selected based on a time stamp comparison such that the previous digital image 200a has been captured at the same time of day as the current digital image 200b, or closest in time to the current digital image 200b.

In further examples, which previous digital image 200a to select is based on illumination, image quality, etc. Also combination of such parameters is envisioned. For example, in order to get the best image quality for optimizing the matching of the keypoints, the previous digital image 200a as well as the current digital image 200b can be extracted within a certain time interval, for example, during daytime. Further, similar time stamps could yield most similar image quality and illumination with respect to the previous digital image 200a and the current digital image 200b.

Aspects of how the coordinate of each at least one annotated point a<NUM>, a<NUM> can be updated will now be disclosed. In some embodiments, the coordinate of each at least one annotated point <MAT> is updated by application of a homography matrix to the coordinate of each at least one annotated point <MAT> in the current digital image 200b. The homography matrix is dependent on the identified amount of movement and the camera homography.

Further aspects of further possible actions taken by the digital image processing device <NUM> will now be disclosed. In some examples, an alarm is triggered if the camera movement causes any annotated point <MAT> to move out from the image. That is, in some embodiments, the digital image processing device <NUM> is configured to perform (optional) step S108:
S108: A notification is issued in case updating the coordinate of each at least one annotated point <MAT> causes any of the at least one annotated point <MAT> to have a coordinate outside the current digital image 200b.

Thereby, if a surveillance region of interest has moved out from the FoV <NUM>, then an alarm can be triggered.

In some aspects, the herein disclosed embodiments can be described in two stages; a feature extraction stage and a calibration stage. The feature extraction stage involves the processing of the previous digital image whereas the calibration stage is applied after camera movement.

Aspects of the feature extraction stage will now be disclosed with reference to the flowchart of <FIG>.

S201: At least one annotated point is set in the digital image.

Each such annotated point might represent a corner of an ROI or an end-point of a virtual line.

S202: Keypoints are searched for in the digital image.

The keypoints might be searched for using a keypoint detector. Non-limiting examples of keypoint detectors are a Harris corner detector, a scale-invariant feature transform (SIFT) detector, and descriptors based on convolutional neural network (CNN). The keypoint detector identifies the coordinates of each keypoint as well as an associated descriptor (or feature vector). In some non-limiting examples the descriptor is provided as a <NUM> to <NUM> variable vector. The descriptors make each of the keypoints distinct and identifiable from the other keypoints. Without loss of generality, the total number of keypoints found is denoted K.

S203: The most appropriate keypoints (descriptors) are selected.

Step S203 might be implemented in terms of a number of sub-steps, or sub-routines.

M ≤ K discriminative keypoints are selected by ranking their similarity. In one example, the similarity between a pair of two keypoints is computed from the Euclidean distance between the descriptors (or feature vectors) of the two keypoints. If the Euclidean distance is smaller than a pre-set threshold distance, this pair of keypoints are defined as similar, and therefore discarded.

The M keypoints may be divided randomly into g subsets Sg. Each such subset consists of N keypoints from the original M keypoints, where N ≪ M. In some examples, at least <NUM> keypoints are selected, that is N ≥ <NUM>. The N keypoints are selected as having the minimum distances, in terms of coordinates, to the image centre and/or the annotated points, as defined by the minimization criterion in (<NUM>): <MAT> where the weighted distance, Sk, is defined as: <MAT>.

In (<NUM>), d(ki, aj) is the Euclidean distance, in terms of coordinates, between each keypoint ki to each annotated point aj in the digital image, and d(ki, C) is the Euclidean distance, in terms of coordinates, between each keypoint ki to the image center C, and wk and wc are their respective weights. Further, J denotes the total number of annotated points. The weights can be used to either prioritize keypoints closer to the annotation points, or keypoints closer to the image center. The minimization criterion in (<NUM>) is used because the closer the distance, in terms of coordinates, between the keypoints and the annotation points in the digital image, the less the probability is for the keypoints to get lost, or even be moved out from the digital image, when the camera <NUM> is moved.

Further, the distance between each two keypoints ki, kj in each subset might fulfil: <MAT>.

The summation of distances d(ki, kj) between each two keypoints is larger than a threshold Dt, which avoids the possibility of keypoints converging together. This can be implemented by different techniques, for example, the RANdom SAmple Consensus (RANSAC) algorithm.

For example, as illustrated in <FIG>, keypoints k<NUM>-k<NUM> have smaller distance, in terms of coordinates, to the annotated points and the image centre than keypoints k<NUM> and k<NUM>, and thus keypoints k<NUM>-k<NUM> are selected whilst keypoints k<NUM> and k<NUM> are discarded.

In surveillance applications, the locations of the annotated points and the image centre are usually the most important positions.

S204: The coordinates and descriptors (feature vectors) of the selected keypoints, as well as the coordinates of the annotated points, are stored.

Also camera settings, e.g. the camera focal length might be stored to enable calibration, or normalization, of the camera view, etc..

Aspects of the calibration stage (as applied after the camera <NUM> has been moved or replaced) will now be disclosed with reference to the flowchart of <FIG>.

S301: The coordinates of the annotated points and the keypoints (with their coordinates and descriptors) of a previous digital image of the same scene are obtained.

S302: Keypoints in the current image are identified. How to identify the keypoints in the current image has been disclosed above with reference to the description of <FIG>.

S303: Matching keypoints are identified using feature vectors determined from the keypoints <MAT> in the current digital image 200b and the keypoints <MAT>,in the previous digital image 200a.

S304: The homography matrix is determined from the matching keypoints.

In this respect, the homography matrix is applied in order to account for camera movement, in terms of camera rotation and/or translation, between the previous digital image and the current digital image. The digital images are assumed to be normalized with respect to each other. Normalization can be achieved using information of the camera focal length used when the previous digital image was captured, and the camera focal length used when the current digital image was captured. In some non-limiting examples, the homography matrix H is computed from the matched annotated points by using Single Value Decomposition (SVD). Further aspects of how the homography matrix can be determined will be disclosed below.

S305: The homography matrix is applied to the at least one annotated point in the current digital image.

The updated coordinates, denoted A', of the at least one annotated point in the current digital image is then determined from the coordinates, denoted A, of the corresponding at least one annotated point in the previous digital image according to: <MAT> where H denotes the homography matrix.

Further aspects of how the homography matrix H can be determined will now be disclosed.

Assume, without loss of generality, that there are four pairs of corresponding annotated points. Denote by a<NUM>, a<NUM>, a<NUM>, a<NUM> the annotated points in the previous digital image and denote by <MAT> the annotated points in the current digital image. Each annotated point has a respective x-coordinate and a respective y-coordinate. Assume further that the focal length is normalized with respect to the previous digital image and the current digital image. The homography matrix H can then be computed as follows.

First, determine the following <NUM>-by-<NUM> matrices: <MAT> where the indices i = <NUM>, <NUM>, <NUM>, <NUM> correspond to the four annotated points. Then stack all these <NUM>-by-<NUM> matrices into one matrix P, which fulfils: <MAT>.

Expression (<NUM>) is thus equivalent to: <MAT>.

Assume further that the matrix P has an SVD decomposition as follows: <MAT> where the columns of V correspond to the eigenvectors of PTP and yield a solution of h. The homography matrix H can be reshaped from h. In order to improve homography matrix H, by minimizing the estimation error, a cost function could be applied. By knowing the homography matrix H, the projected coordinates of the point a(x, y) from the previous digital image to the point a'(x, y) from the current digital image is thus given by: <MAT>.

The coordinates for annotated point pi' can thus be found by inserting x = xi, y = yi, <MAT>, and <MAT> in the expression (<NUM>) for i = <NUM>, <NUM>, <NUM>, <NUM>.

<FIG> schematically illustrates a first scenario where the herein disclosed embodiments can be applied. At (a) is illustrated a previous digital image 600a of a scene <NUM> as captured by a camera <NUM> of a video surveillance system. The scene comprises a building <NUM>. For privacy reasons, the windows 630a:630f (not shown in <FIG>) of the building are to be excluded from the surveillance. Six different privacy masks 640a:640f, each covering a window 630a:630f, are therefore applied. The lower-left corners of the privacy masks are defined by annotated points a<NUM>:a<NUM>. At (b) is illustrated a current digital image 600b of the scene as captured by the camera <NUM> after camera movement. As noted at (b), since the annotated points a<NUM>:a<NUM> have the same coordinates as in the previous digital image 600a, the camera movement causes the privacy masks to be moved, thus partly revealing the windows 630a:630f. At (c) is illustrated the current digital image 600b' of the scene as captured by the camera <NUM> after camera movement upon application of at least some of the herein disclosed embodiments for updating the coordinates of the annotated points <MAT> in the current digital image due to the camera movement. As can be seen, the updating causes the privacy masks to be moved back to their correct positions, thus again properly covering the windows 630a:630f.

<FIG> schematically illustrates a second scenario where the herein disclosed embodiments can be applied. At (a) is illustrated a previous digital image 700a of a scene as captured by a camera <NUM> of a video surveillance system. The scene comprises buildings <NUM> and a parking lot <NUM>. The parking lot defines a ROI <NUM> and is under surveillance. An ROI is therefore defined based on the corners of the parking lot. The corners of the ROI are defined by annotated points (represented by the single annotated point a<NUM>). At (b) is illustrated a current digital image 700b of the scene as captured by the camera <NUM> after camera movement. As noted at (b), since the annotated points (again represented by the single annotated point a<NUM>) have the same coordinates as in the previous digital image 700a, the camera movement causes the ROI <NUM> to be moved, thus partly excluding some parts of the parking lot <NUM> from surveillance. At (c) is illustrated the current digital image 700b' of the scene as captured by the camera <NUM> after camera movement upon application of at least some of the herein disclosed embodiments for updating the coordinates of the annotated points (represented by the single annotated point <MAT>) in the current digital image due to the camera movement. As can be seen, the updating causes the ROI <NUM> to be moved back to its correct position, thus again properly covering the parking lot <NUM>.

<FIG> schematically illustrates, in terms of a number of functional units, the components of an image processing device <NUM> according to an embodiment. Processing circuitry <NUM> is provided using any combination of one or more of a suitable central processing unit (CPU), multiprocessor, microcontroller, digital signal processor (DSP), etc., capable of executing software instructions stored in a computer program product <NUM> (as in <FIG>), e.g. in the form of a storage medium <NUM>. The processing circuitry <NUM> may further be provided as at least one application specific integrated circuit (ASIC), or field programmable gate array (FPGA).

Particularly, the processing circuitry <NUM> is configured to cause the image processing device <NUM> to perform a set of operations, or steps, as disclosed above. For example, the storage medium <NUM> may store the set of operations, and the processing circuitry <NUM> may be configured to retrieve the set of operations from the storage medium <NUM> to cause the image processing device <NUM> to perform the set of operations.

The image processing device <NUM> may further comprise a communications interface <NUM> at least configured for communications with the camera <NUM>, potentially with other functions, nodes, entities and/or devices, such as functions, nodes, entities and/or devices of a video surveillance system. As such the communications interface <NUM> may comprise one or more transmitters and receivers, comprising analogue and digital components. The processing circuitry <NUM> controls the general operation of the image processing device <NUM>, e.g., by sending data and control signals to the communications interface <NUM> and the storage medium <NUM>, by receiving data and reports from the communications interface <NUM>, and by retrieving data and instructions from the storage medium <NUM>. Other components, as well as the related functionality, of the image processing device <NUM> are omitted in order not to obscure the concepts presented herein.

The image processing device <NUM> may be provided as a standalone device or as a part of at least one further device. For example, the image processing device <NUM> and the camera <NUM> might be part of a video surveillance system.

A first portion of the instructions performed by the image processing device <NUM> may be executed in a first device, and a second portion of the of the instructions performed by the image processing device <NUM> may be executed in a second device; the herein disclosed embodiments are not limited to any particular number of devices on which the instructions performed by the image processing device <NUM> may be executed. Hence, the methods according to the herein disclosed embodiments are suitable to be performed by an image processing device <NUM> residing in a cloud computational environment. Therefore, although a single processing circuitry <NUM> is illustrated in <FIG> the processing circuitry <NUM> may be distributed among a plurality of devices, or nodes. The same applies to the computer program <NUM> of <FIG>.

Claim 1:
A method for updating a coordinate of an annotated point ( <MAT>) in a digital image (200b) due to camera movement, the method being performed by an image processing device (<NUM>), the method comprising:
obtaining (S102) a current digital image (200b) of a scene (<NUM>), the current digital image (200b) having been captured by a camera (<NUM>) subsequent to movement of the camera (<NUM>) relative the scene (<NUM>), wherein the current digital image (200b) is associated with at least one annotated point ( <MAT>) and wherein each at least one annotated point ( <MAT>) has a respective coordinate in the current digital image (200b);
identifying (S104) an amount of the movement of the camera by comparing position indicative information in the current digital image (200b) to position indicative information in a previous digital image (200a) of the scene (<NUM>), the previous digital image (200a) having been captured prior to the movement of the camera (<NUM>), wherein the position indicative information in the current digital image (200b) is represented by a first set of keypoints ( <MAT>) extracted from the current digital image (200b), wherein the position indicative information in the previous digital image (200a) is represented by a second set of keypoints (k<NUM>:k<NUM>) extracted from the previous digital image (200a), wherein the second set of keypoints (k<NUM>:k<NUM>) is a subset of all keypoints (k<NUM>:k<NUM>) extractable from the previous digital image (200a), wherein the previous digital image (200a) is associated with at least one annotated point (a<NUM>, a<NUM>) having a respective coordinate in the previous digital image (200a), and wherein the second set of keypoints (k<NUM>:k<NUM>) is restricted to only include keypoints (k<NUM>:k<NUM>) located within a predefined radius from the coordinate of the at least one annotated point (a<NUM>, a<NUM>) in the previous digital image (200a); and
updating (S106) the coordinate of each at least one annotated point ( <MAT>) in accordance with the identified amount of movement of the camera and a camera homography.