Patent Description:
Several types of devices are known from the state of the art which, by exploiting algorithms based on neural networks, identify and recognize targets present in an environment. These algorithms associate a boundingbox with the different targets and subsequently employ segmentation algorithms, associated with neural networks, to recognize their contours. In certain cases, depth information is also known to be acquired in order to associate spatial coordinates with the different recognized targets for later use.

Publication "<NPL> presents a neural network based visual perception framework for autonomous apple harvesting from RGB-D images. Publication "<NPL>et al. presents a deep leaning - based method for automatically detecting from RGB-D images pairs of humans in a crowded scenario who are not adhering to the social distance constraint.

Known algorithms based on neural networks and used for recognizing and defining target contours require a high computational cost that makes them difficult to manage.

In addition, the boundingboxes associated with the different targets may have deviations from the target, incorrect orientations with respect to the target, and/or dimensions that are larger or smaller than the target. Such boundingbox incorrections cause an incorrect handling of the acquired information and require computationally expensive segmentation algorithms to correct them.

In addition, it should be noted that depth information is handled passively by the algorithms, not guaranteeing the correct association of the depth information with each pixel as well as the difficulty in handling undefined depths. In fact, it happens that "masking" phenomena occur during the acquisition of depth values, which are due for example to the "disparity" between two successive images or acquisition problems mainly at the edges thereof. Such maskings cause the acquisition of depth information to be missed. This fact leads to an incorrect association of spatial coordinates with the target and therefore to an incorrect definition of the target itself.

The object of the present invention is to realise a method and a device capable of overcoming the above mentioned drawbacks of the prior art.

In particular, the object of the present invention is to provide a method capable of improving the handling of the acquired depth information as well as the identification of targets and their mutual position.

The specified technical task and the specified purposes are substantially achieved by a method for monitoring targets and an imaging device, comprising the technical characteristics set out in one or more of the appended claims.

Advantageously, the method and the device of the present invention allows to overcome the lack of information related to depth.

Advantageously, the method and the device of the present invention make it possible to refine and orient the boundingboxes associated with the recognized targets while reducing the necessary computational cost.

Advantageously, the method and the device of the present invention enable to define the contours of the recognized targets while reducing the necessary computational cost.

Advantageously, the method and the device of the present invention allow the relative distance between several targets to be determined.

Further features and advantages of the present invention will become clearer from the indicative and therefore non-limiting description of a preferred but not exclusive embodiment of a method for monitoring targets and an imaging device as illustrated in the accompanying drawings:.

Even when not explicitly highlighted, the individual features described with reference to the specific embodiments must be considered as accessories and/or exchangeable with other features, described with reference to other embodiments.

The present invention relates to a method for monitoring targets <NUM> arranged in an environment <NUM>. Preferably, the method for monitoring targets <NUM> is carried out by means of an imaging device <NUM>, described in detail below. The method subject-matter of the present invention allows to detect targets <NUM> present in the environment <NUM> and to correct errors due to the acquisition of data, preferably associated with the acquired images and depth information.

For the purposes of the present invention, a target <NUM> means any object, subject and entity present in an environment <NUM>. For example, target <NUM> could refer to fruit and vegetable products and/or people and/or animals.

For the purposes of the present invention, environment <NUM> means any place framed by the imaging device <NUM> such as an orchard and/or a square or a street.

The method for monitoring targets <NUM> subject-matter of the present invention comprises the following steps, carried out according to a preferred embodiment and shown in <FIG> and in subsequent <FIG> and <FIG>.

The method for monitoring targets <NUM> comprises a step of defining <NUM> a reference system Rif with respect to the imaging device <NUM>. In particular, the reference system Rif defines a plane of the image PI, preferably associated with the imaging device <NUM>, and a depth dimension DP extending from the plane of the image PI, preferably from the imaging device <NUM>. More preferably, the reference system Rif defines the plane of the image PI on two-dimensional images acquired by the imaging device <NUM> and the depth dimension DP along which a distance value D from the plane of the image PI is measurable. Even more preferably, the reference system Rif allows to define spatial coordinates X and Y on the plane of the image PI and Z on the depth dimension DP.

The method for monitoring targets <NUM> comprises a step of acquiring <NUM> by means of the imaging device <NUM> a first image stream and a second image stream. In particular, the first image stream is related to the environment <NUM>. Preferably, the first image stream comprises two-dimensional colour images of the environment <NUM>. It should be noted that each image in the first image stream is defined by a plurality of pixels. In other words, the first image stream comprises a plurality of temporally successive frames. As regards the second image stream, it is related to distance values D of the pixels from the reference system Rif. In other words, the second image stream detects distance values D from the reference system Rif to the targets <NUM> present in the environment <NUM>. Preferably, the second image stream comprises a plurality of distance values D that can be associated with the image pixels of the first image stream. More preferably, the second image stream comprises a plurality of frames each comprising a plurality of distance pixels each associated with a distance value D, if detected. In particular, the second image stream comprises a plurality of temporally successive frames comprising distance values.

It should be noted that each image in the first image stream is associated with the plane of the image PI and each distance value D is associated with the depth dimension DP.

Preferably, the step of acquiring <NUM> provides for acquiring the first image stream and the second image stream present in the Field of View (FOV) of the imaging device <NUM>.

The method for monitoring targets <NUM> comprises a step of sending <NUM> the first image stream and the second image stream to a data processing unit <NUM>. Preferably, the step of sending <NUM> provides for sending, from the imaging device <NUM> to the data processing unit <NUM> in signal communication with the imaging device <NUM>, the first image stream and the second image stream acquired. More preferably, the method for monitoring targets <NUM> comprises a step of storing <NUM> the first image stream and the second image stream in the data processing unit <NUM> at least temporarily. In particular, the temporary storage of the first image stream and of the second image stream optimises the computational resources for the subsequent steps.

In accordance with a preferred embodiment, the method for monitoring <NUM> comprises a step of synchronising <NUM> the first image stream and the second image stream. Preferably, the synchronisation step <NUM> provides for associating each pixel of the images of the first image stream with a corresponding distance value D of the second image stream.

Preferably, the method for monitoring targets <NUM> comprises a step of storing <NUM> the synchronisation between the first image stream and the second image stream carried out at the step of synchronising <NUM>.

The method for monitoring targets <NUM> comprises a step of recognizing <NUM> one or more targets <NUM> present in the images of the first image stream. Preferably, the step of recognizing <NUM> provides for analysing each image of the first image stream to recognize the targets <NUM> present in the relative image. It should be noted that the step of recognizing one or more targets <NUM> provides for recognizing one or more targets <NUM> with respect to a background.

The step of recognizing one or more targets <NUM> present in the images of the first image stream is carried out by means of an algorithm based on neural networks and associated with the environment. Preferably, the algorithm based on neural networks is optimised in the size, in particular quantized. More preferably, the algorithm based on neural networks is trained to recognize one or more targets <NUM> present in the images of the first image stream. Even more preferably, the algorithm based on neural networks is configured, in other words trained, to recognize specific targets <NUM> as a function of the environment <NUM>. For example, if the first image stream is acquired in an orchard, the algorithm based on neural networks can be trained to recognize the species of fruit present in the orchard. Or, for example, if the first image stream is acquired in a street or square, the algorithm based on neural networks is trained to recognize people.

Advantageously, the use of an algorithm based on neural networks configured to recognize targets <NUM> in a specific environment <NUM> reduces the computational cost required.

The step of recognizing <NUM> comprises the steps of geolocating the imaging device <NUM> in order to recognize the environment in which the imaging device <NUM> is inserted. Subsequently, once the environment has been recognized it comprises a step of loading the algorithm based on neural networks suitable for the recognized environment.

Advantageously, loading a specific algorithm enables the adaptability of the method to different types of environment while maintaining a reduced computational cost.

The method for monitoring targets <NUM> comprises a step of associating <NUM> a boundingbox <NUM> with each recognized target <NUM>. In particular, the step of associating <NUM> provides for the insertion of a recognized target <NUM> into a boundingbox <NUM>. In detail, one or more boundingboxes <NUM> are identified on each image of the first image stream as a function of the recognized targets <NUM>. Preferably, each boundingbox is associated with boundingbox coordinates Xb, Yb, Zb on the plane of the image as a function of the reference system Rif. More preferably, each boundingbox is associated with spatial coordinates with respect to the reference system.

For the purposes of the present invention, boundingbox <NUM> means a quadrilateral, for example a square or rectangle, parellelogram, which contours the target to which relative pixels of the images of the first image stream are associated. It should be noted that each boundingbox <NUM> has a plurality of sides, preferably four, to which pixels are associated.

For the purposes of the present invention, the lengths of the sides, for example height H and width L, of the boundingboxes <NUM> are measured as a function of the pixels associated therewith as a function of the reference system Rif.

Preferably, the boundingbox coordinates Xb, Yb, Zb comprise the position of the pixel on the plane of the image associated with opposite angles of the boundingbox with respect to the diagonal. Alternatively, the boundingbox coordinates comprise only the position of an angle and values of widths L and height H in pixels of the boundingbox <NUM>.

More preferably, each boundingbox <NUM> is associated with an outer perimeter 50a contouring the identified target <NUM>.

It should be noted that each boundingbox <NUM> comprises a plurality of pixels preferably delimited within the outer perimeter 50a.

In accordance with a preferred embodiment, the step of associating a boundingbox <NUM> also provides for associating a recognition label and a confidence level to each target. Specifically, the confidence level is associated with how certain of the object recognition the recognition carried out in the recognition step <NUM> is and is defined as a function of the algorithm based on neural networks.

The method for monitoring targets <NUM> comprises a step of calculating <NUM> a significant distance value Ds of the boundingboxes <NUM> with respect to the reference system Rif as a function of the second image stream.

Preferably, the step of calculating <NUM> a significant distance value Ds comprises the steps of checking <NUM> the availability of a distance value D for each pixel of the boundingboxes <NUM>. A subsequent step of associating <NUM> for each pixel of each boundingbox <NUM> a respective distance value D if available. Specifically, the step of associating <NUM> provides for checking the availability of a distance value D for each pixel associated with the boundingbox <NUM>. Thereafter, the step of calculating <NUM> a significant distance value Ds comprises the step of determining <NUM> each significant distance value Ds as a function of the pixels associated with a distance value D. Preferably, the step of determining <NUM> each significant distance value Ds provides for discarding the pixels of the image of the first image stream lacking the relative distance value D. If, the number of pixels of the image of the first image stream lacking the corresponding distance value D exceed a minimum threshold value, the determining step <NUM> provides for discarding the relative boundingbox <NUM>.

It should be noted that in accordance with a preferred embodiment of the present invention, the calculating step <NUM> provides for calculating the significant distance of each boundingbox having a confidence level higher than a threshold confidence value. Specifically, the calculating step <NUM> provides for a first step of selecting each boundingbox <NUM> to which a confidence level higher than a threshold confidence value is associated.

More preferably, the method for monitoring targets <NUM> comprises a step of storing <NUM> the significant distance values Ds calculated at step <NUM>.

In accordance with a preferred embodiment of the method, illustrated in <FIG>, the significant distance Ds corresponds to the mean or median of the distances D associated with the pixels of the respective boundingbox <NUM> of the images of the first image stream.

The mean or the median represents the reference distance of the detected object as a whole. This measure is taken into account by the method when assessing the dimensions and/or the distance between the other recognized targets <NUM>.

Preferably, the method for monitoring targets <NUM> comprises a step of orienting <NUM> the boundingboxes <NUM> on the recognized target.

Advantageously, the correct orientation of the boundingbox <NUM> on the recognized target improves and facilitates the subsequent steps.

The orientation step <NUM> comprises a step of associating <NUM> the relative outer perimeter 50a to at least one boundingbox <NUM>. Subsequently, the orientation step <NUM> comprises the step of selecting <NUM> at least two perimeter points <NUM>, <NUM> on said outer perimeter 50a. It should be noted that perimeter points <NUM>, <NUM> refer to the spatial coordinates on the plane of the image PI of the pixels associated with them. Preferably, the perimeter points <NUM>, <NUM> are substantially in positions facing the outer perimeter 50a. Once the points have been selected, the orientation step <NUM> comprises a step of determining <NUM> a reference axis X-X as a function of the target <NUM> recognized and associated with the related boundingbox <NUM>. Preferably, the determination of the reference axis X-X depends on the recognized target <NUM>. For example, if the target were an apple, the reference axis could be the one associated with the core itself. Finally, the orientation step <NUM> comprises the step of orienting <NUM> the outer perimeter 50a of the boundingbox <NUM> on the target <NUM> as a function of the reference axis X-X and of the perimeter points <NUM>, <NUM>. Preferably, the orientation step <NUM> provides for rotating and/or deforming the boundingbox <NUM> as a function of the perimeter points <NUM>, <NUM> and of the reference axis X-X.

In accordance with a preferred embodiment, the method for monitoring targets <NUM> comprises a step of defining in plan the contours of a recognized target <NUM> as a function of the first image stream. In particular, the step of defining in plan the contours <NUM> provides for defining the contours of a target <NUM> within a boundingbox <NUM> on the image of the first image stream as well as on the two-dimensional colour image. In detail, the step of defining in plan the contours <NUM> allows the contours to be defined on the plane of the image associated with the first image stream. In other words, the step of defining in plan the contours <NUM> allows the target <NUM> to be distinguished from the background. Preferably, the step of defining in plan the contours <NUM> comprises the step of selecting <NUM> at least one boundingbox <NUM> with which the outer perimeter 50a is associated. Subsequently, the step of defining in plan the contours <NUM> comprises a step of selecting <NUM> a starting area <NUM> on the recognized target <NUM> within the outer perimeter 50a of the boundingbox <NUM>. Preferably, the starting area <NUM> is substantially in the geometric centre of the outer perimeter 50a of the boundingbox <NUM> and comprises one or more image pixels of the first image stream. In this way, the selection of a starting area <NUM> makes it possible to identify one or more distinctive marks of the target such as for example the colour. After selecting the starting area <NUM>, the step of defining in plan the contours <NUM> comprises the step of associating <NUM> with the starting area <NUM> first plan target parameters associated with the first image stream. In particular, the first plan target parameters are related to the distinctive marks of the target such as its colour. The step of defining in plan the contours <NUM> comprises the steps of:.

In this way, the step of defining in plan the contours <NUM> allows a variation between the first plan target parameters and the second plan target parameters to be recognized. Subsequently, the step of defining in plan the contours <NUM> comprises the step of comparing <NUM> the first plan target parameters with the second plan target parameters in order to detect variations beyond the first threshold parameters and thus generate <NUM> a contour value if the comparison exceeds the first threshold parameters. Specifically, the step of generating <NUM> a contour value provides for generating this value when the second threshold parameters are associated with the background rather than with the target <NUM>. Finally, the step of defining in plan the contours <NUM> comprises the step of delineating <NUM> the contour <NUM> of the target <NUM> as a function of the contour values.

In accordance with a preferred embodiment alternative to the previous one but combinable therewith, the method for monitoring targets <NUM> comprises a step of defining in depth the contours of a recognized target <NUM> as a function of the second image stream. It should be noted that the step of defining in depth the contours of a recognized target <NUM> comprises substantially the same steps as the step of defining in plan the contours of a recognized target <NUM> but using the information associated with the distance D of the pixels from the reference system Rif.

Preferably, the step of defining the contours <NUM> in depth comprises the steps of:.

Advantageously, the step of defining in plan the contours of a recognized target <NUM> and the step of defining in depth the contours of a recognized target <NUM> allow the contours of recognized targets to be defined while reducing the required computational cost.

Advantageously, the definition of a boundingbox <NUM> associated with a recognized target speeds up the definition of the contours <NUM> of the targets <NUM> in computational terms.

Preferably, the step of defining in plan the contours of a recognized target <NUM> and the step of defining in depth the contours of a recognized target <NUM> are performed by computer vision-based algorithms and/or depth-based algorithms. For example, a computer vision-based algorithm may be a random walker algorithm. It should be noted that the step of defining in plan the contours of a recognized target <NUM> and the step of defining in depth the contours of a recognized target <NUM> can also be based on specific neural networks for segmentation.

In accordance with a preferred embodiment, the step of orienting the boundingboxes <NUM> is carried out as a function of the definition of the contour <NUM> of the target. In particular, the boundingbox <NUM> is oriented and redefined on the basis of the contours <NUM> of the relative target <NUM>.

In accordance with a preferred embodiment, the method for monitoring targets <NUM> comprises a step of calculating <NUM> the dimensions of the recognized targets <NUM> as a function of the boundingbox <NUM> with which the relative pixels of the first image stream and/or the distances D of the second image stream are associated. Specifically, the step of calculating <NUM> provides for calculating the dimensions, preferably in millimetres, of each boundingbox <NUM>. In detail, the dimensions refer to the lengths in pixels of the sides of a boundingbox <NUM>. It should be noted that the calculating step <NUM> can be carried out both on the boundingbox <NUM> associated with the target <NUM> at step <NUM> and on the boundingbox <NUM> oriented at step <NUM> as well as on the boundingbox <NUM> oriented following the definition of the contour <NUM>.

Preferably, the calculating step <NUM> provides for associating to the boundingbox <NUM> the dimensions of the boundingbox, for example height H and width L, and optionally the significant distance Ds, in the present embodiment represented by the median or the mean.

Advantageously, the dimensions of the boundingbox <NUM> associated with specific targets make it possible to define limit thresholds for certain operations as well as further characteristics of the target. In a first case, the volume and the weight of the target <NUM> can be calculated as a function of its specific weight identified by the environment in which the imaging device <NUM> is placed. For example, if the imaging device <NUM> were positioned in an apple orchard, the algorithms employed and trained on apple recognition could comprise additional data such as the average weight of the apples with certain dimensions and/or the specific weight of the apples themselves and determine the weight of the targets by means of known algorithms. In a second case, it is possible to intervene if the calculated dimensions exceed a threshold limit, such as when grinding gravel in a quarry and to have a control over the grain size, when measuring the dimensions of artificial snowflakes or when measuring farmed fish.

More preferably, the method for monitoring targets <NUM> comprises a step of storing <NUM> the data generated in steps <NUM>, <NUM>, <NUM> and <NUM> such as the recognized targets, the boundingboxes, the target contours and the target dimensions produced in steps <NUM>, <NUM>, <NUM> and <NUM>.

In accordance with a preferred embodiment illustrated in <FIG> and alternative to the embodiment illustrated in <FIG> but combinable, the method for monitoring targets <NUM> comprises a step of determining the relative distance between two or more targets identified by a boundingbox <NUM>.

Advantageously, the relative distance between more targets <NUM> makes it possible to take into account, for example, the correct distance between two or more individuals or to evaluate density values of fruit clusters in an orchard.

The step of determining the relative distance between two or more targets identified by a boundingbox <NUM> comprises the step of defining <NUM> for each boundingbox <NUM> at least one significant point <NUM>. Preferably, the significant point <NUM> is identifiable with the centroid of the relative boundingbox <NUM>. In particular, the step of defining <NUM> for each boundingbox <NUM> at least one significant point <NUM> provides for associating the significant distance value Ds relative to the boundingbox to the significant point <NUM>. In accordance with the present embodiment, the significant distance value Ds is equal to the average distance of the significant point <NUM> from the reference system Rif. Subsequently, the step of defining <NUM> for each boundingbox <NUM> at least one significant point <NUM> provides for assigning two spatial plan coordinates A, B to the significant point <NUM> with respect to the reference system Rif. In detail, the step of defining <NUM> for each boundingbox <NUM> at least one significant point <NUM> associates three spatial coordinates to each significant point <NUM>.

For the purposes of the present invention, the centroid can be replaced by other significant points <NUM> of the boundingbox as a function of the target. For example, in the case of measuring social distancing, the significant point may be the position of a subject's feet, head or upper torso.

Upon identifying the significant point <NUM> and associating significant distance Ds and spatial plan coordinates A, B, the step of determining the relative distance Dr between two or more targets <NUM> identified by a boundingbox <NUM> comprises the step of calculating <NUM> the relative distance Dr between the significant points <NUM> of each boundingbox <NUM>. The step of determining the relative distance Dr between two or more targets identified by a boundingbox <NUM> comprises the step of generating <NUM> an evaluation signal as a function of the relative distance Dr between the significant points <NUM> of each boundingbox <NUM> and threshold evaluation values.

Preferably, the method for monitoring images <NUM> comprises a step of generating an alarm signal if an evaluation signal linked to the trespassing of minimum threshold values by the relative distance Dr is generated. For example, when the method is applied to social distancing. Alternatively, the method for monitoring images <NUM> comprises a step of processing the evaluation signal in order to extrapolate information from the targets e.g. fruit density in an orchard.

In accordance with a preferred embodiment, the step of calculating the relative distance Dr between significant points of each boundingbox <NUM> comprises the sub-steps of determining 212a a relative distance Dr in the space between two significant points <NUM>. Preferably, the relative distance is calculated by defining a vector of spatial coordinate values as a function of the reference system Rif for each significant point <NUM>. Subsequently, step 212a provides for calculating the relative distance between two boundingboxes by means of known algorithms, such as using the Euclidean distance (shown below).

Preferably, the relative distance Dr is calculated between two different boundingboxes <NUM> only once. More preferably, a matrix of relative distances is constructed using the vectors of the spatial coordinate values Dr. In detail, the values of the main diagonal and of the lower or upper triangle are set at higher values than the threshold values avoiding the repetitive calculation of relative distances Dr between two significant points <NUM>. In this way, the calculation of the relative distances Dr is optimised as the complexity of the calculation itself is reduced. It should be noted that the relative distance matrix has columns and rows equal to the number of boundingboxes <NUM> for which the relative distance Dr is calculated.

Preferably, the step of calculating the relative distance between significant points of each boundingbox <NUM> comprises the sub-step of defining 212b a calculation matrix by means of the squares of the relative distances Dr between significant points <NUM> of different boundingboxes <NUM>. For example, using the squared Euclidean distance formula: <MAT>.

Finally, the step of calculating the relative distance Dr between significant points <NUM> comprises the sub-step of comparing 212c the square of the distance Ds with the square of the evaluation threshold values.

It should be noted that in accordance with a preferred embodiment of the present invention, the step of determining the relative distance Dr between two or more targets identified by a boundingbox <NUM> provides for calculating the relative distances Dr between boundingboxes <NUM> associated with recognized targets <NUM> and associated with objects and/or subjects of interest. For example, if the monitoring method were used in the evaluation of interpersonal distance, the targets of interest would comprise people.

More preferably, the method for monitoring targets <NUM> comprises a step of storing <NUM> the data generated by step <NUM> such as the relative distances Dr, the evaluation and alarm signals.

In accordance with a preferred embodiment, the method for monitoring targets <NUM> is applicable to targets <NUM> that are stationary and/or moving in an environment <NUM> with respect to the reference system Rif as well as with respect to the imaging device <NUM>. It should be noted that the method for monitoring targets <NUM> can be applied by moving the imaging device <NUM> with respect to stationary and/or moving targets <NUM>.

For the purposes of the present invention, the method for monitoring images <NUM> comprises a debugging/testing step in order to control the steps of the method. In particular, the debugging/testing step comprises the steps of:.

Advantageously, the debugging/testing step makes it possible to improve and optimise the method for monitoring targets <NUM>.

It should be noted that this debugging/testing step is applicable to each step of the method for monitoring targets <NUM>.

An imaging device <NUM> configured to implement the method for monitoring targets <NUM> is a further object of the present invention.

The imaging device comprises a camera <NUM> configured to acquire the first image stream and the second image stream. In particular, the camera <NUM> is configured to film the environment in which it is placed.

In accordance with a preferred embodiment, the camera <NUM> comprises optical sensors, for example RGB for colour filming of the first image stream, and one or more infrared sensors for infrared filming for the second image stream. Preferably, the optical and infrared sensors comprise a shutter of the global shutter type, free from "creep" phenomena when filming in motion. The optical and infrared sensors can have a specific Field of View (FOV) representing the horizontal and vertical viewing angle that each sensor supports. In other words, the optical sensors and the infrared sensors have a FOV capable of determining the filmed area given a distance of the filmed environment.

In this way, the use of optical and infrared sensors allows, thanks to the method, to generate a combined image stream in which each pixel of the first image stream is associated with a distance value, if available according to the steps described above.

Preferably, the camera <NUM> is configured to work at an adjustable frame rate and as a function of the environment in which the camera <NUM> is inserted, and as a function of any movements of the camera with respect to the environment.

In accordance with the present invention, the camera is of the RGB-D type.

In accordance with an alternative embodiment, the imaging device comprises a camera for two-dimensional colour filmings (first image stream) and depth sensors (second image stream). Advantageously, the method for monitoring <NUM> allows the combination of the first and second image stream.

The imaging device <NUM> comprises a data processing unit <NUM> in signal communication with the camera <NUM>. The data processing unit <NUM> is configured to receive the first image stream and the second image stream and process them by performing the aforesaid steps of the method for monitoring targets <NUM>.

In accordance with a preferred embodiment, the data processing unit <NUM> comprises a memory module (Redis) configured to receive data, such as the first image stream and the second image stream, and processed data such as the boundingboxes <NUM> and to save them. The data processing unit <NUM> comprises a synchronisation module (Frame splitter), in signal communication with the memory module Redis, and configured to synchronise the first image stream and the second image stream. This synchronisation module Framesplitter is also configured to save synchronisations on the memory module Redis. The data processing unit <NUM> further comprises a detection module (Detect), in signal communication with the memory module Redis and configured to recognize targets and associate relative boundingboxes <NUM> to the recognized targets <NUM>. Preferably, the algorithm based on neural networks is resident in the detection module, Detect. More preferably, the detection module Detect is in signal communication with a geolocation module so as to receive a specific algorithm based on neural networks as a function of the environment in which the imaging device <NUM> is inserted. It should be noted that the detection module Detect is configured to save the recognized targets <NUM> and the boundingboxes <NUM> generated in the memory module Redis.

In accordance with the embodiment of <FIG> and <FIG>, the data processing unit <NUM> comprises a caliper module (Caliper) in signal communication with the memory module Redis and configured to calculate the significant distance Ds, orient the boundingboxes <NUM>, define in plan or in depth contours and calculate the dimensions of the recognized targets <NUM>. The caliper module Caliper is also configured to save the data generated in the memory module Redis.

In accordance with the embodiment of <FIG> and <FIG>, the data processing unit <NUM> comprises a measurement module (Measure) in signal communication with the memory module Redis and configured to determine the relative distance Dr between boundingboxes <NUM> and save them in the memory module Redis.

In accordance with a preferred embodiment, the imaging device <NUM> comprises an alarm module (Alarm) in signal communication with the data processing unit <NUM>, preferably with the memory module Redis, and configured to generate evaluation signals and/or alarm signals as a function of the threshold values.

In accordance with a preferred embodiment, the imaging device <NUM> comprises a remote control module in signal communication with a device external to the imaging device <NUM> and associated with a user. Preferably, the remote control module is configured to remotely control the imaging device <NUM> and allow the display of the data saved in the memory module Redis.

It should be noted that the imaging device <NUM> subject-matter of the present invention is advantageously applicable in the fields of social distancing and in agriculture as well as in the dimensional evaluation of targets such as grains of a quarry, fish, and artificial snowflakes or on the attendance of certain environments such as the departments of a supermarket.

Advantageously, the method for monitoring images allows reducing the necessary computational cost and the use of resources by making the imaging device operable in situ, at the EDGE, requiring only a power supply. In particular, given the low cost of computation and of use of resource, the power supply can be supplied directly on site without the imaging device <NUM> being connected to a power line. For example, the power supply needed by the imaging device can be supplied by means of renewable energy, such as a solar panel, or pre-charged batteries.

Claim 1:
Method for monitoring (<NUM>) targets (<NUM>) arranged in an environment (<NUM>) by means of an imaging device (<NUM>), said monitoring method (<NUM>) comprising the steps of:
- defining (<NUM>) a reference system (Rif) with respect to the imaging device (<NUM>)
- acquiring (<NUM>) by means of the imaging device (<NUM>):
a first image stream related to the environment (<NUM>), each image of the first image stream being defined by a plurality of pixels;
- a second image stream related to distance values (D) of the pixels from the reference system (Rif);
- sending (<NUM>) the first image stream and the second image stream to a data processing unit (<NUM>);
- recognizing (<NUM>) one or more targets (<NUM>) present in the images of the first image stream;
- associating (<NUM>) a boundingbox (<NUM>) with each recognized target (<NUM>);
- calculating (<NUM>) a significant distance value (Ds) of the boundingboxes with respect to the reference system (Rif) as a function of the second image stream;
the step of calculating (<NUM>) a significant distance value (Ds) comprising the steps of:
- checking (<NUM>) the availability of a distance value (D) for each pixel of each boundingbox (Box);
- associating (<NUM>) for each pixel of each boundingbox (Box) a respective distance value (D), if available;
- determining (<NUM>) each significant distance value (Ds) as a function of the pixels associated with a distance value (D);
wherein the step (<NUM>) of recognizing one or more targets present in the images of the first image stream is performed by an algorithm based on neural networks resident in the data processing unit (<NUM>) of the imaging device (<NUM>),
characterized in that said algorithm is trained to recognize one or more targets (<NUM>) present in the images of the first image stream related to the environment (<NUM>) where the imaging device (<NUM>) is inserted, said step of recognizing (<NUM>) comprising the steps of:
- geolocating the imaging device (<NUM>) to recognize the environment (<NUM>) in which the imaging device (<NUM>) is inserted;
- loading the algorithm based on neural networks associated with the recognized environment suitable for the recognized environment.