Patent Description:
New technology opens up new opportunities. For instance, the evolution of digital cameras and communication technologies enable monitoring of people to be provided using video surveillance at relatively low cost. This can be particularly useful for elderly people or disabled people, who in this way can enjoy greatly improved quality of life by living in their own home instead of being in a staffed care facility. Video data can also be used e.g. for people counting.

Video surveillance is certainly useful, but privacy issues arise. Hardly anyone enjoys being continuously monitored using video surveillance, for monitoring of when the person needs help.

One way to reduce the privacy concern is to, instead of manual monitoring, use machine learning (ML) models to determine the state of a monitored person. However, also the video-data based ML models need to be trained, which requires video data to be provided to the ML models. Such video data for training sometimes needs to be manually processed as part of the training process, which is a privacy concern for the person captured in the video data.

<CIT> discloses privacy supporting computer vision systems, methods, apparatuses and associated computer executable code. <CIT> discloses a video-based fall risk assessment system. <CIT> discloses protection and recovery of identities in surveillance camera environments. <NPL> an article entitled "On the Role of Data Anonymization in Machine Learning Privacy", discusses how data controllers may anonymize data before releasing them for any data analysis task such as machine learning. Wong Kok-Seng Wong Ks@Vinuni <NPL> an article entitled "A Privacy-Preserving Framework for Surveillance Systems", is a paper proposing dynamic masking to ensure that an individual remains anonymous in a group.

One object is to provide an improved balance between privacy and training data requirements when providing training data based on a data feed capable of depicting people.

According to a first aspect, it is provided a method for enabling training of a machine learning, ML, model, for monitoring a person based on a data feed capable of depicting a person. The method is performed by a training data provider comprising a processor; and a memory storing instructions to be executed by the processor. The method comprises: obtaining a data feed capable of depicting the person; selecting a level of anonymisation, from a plurality of levels of anonymisation; anonymising the data feed according to the selected level of anonymisation, resulting in a processed data feed; transmitting the processed data feed as training data for training a central ML model in a central node; receiving an indication to increase or reduce the level of anonymisation from the central node. The method is repeated, wherein the next iteration of the selecting is based on the indication to increase or reduce the level of anonymisation.

The levels of anonymisation may include in, in order of increasing anonymisation: blurring of face, replacing face with a computer-generated face image, replacing face with a picture of someone else's face, blurring of entire body.

The anonymising may comprise selecting another face of the same gender as the person in the data feed.

The method may further comprise: determining a label associated with the data feed; and including the label in association with the processed data feed.

The label may indicate a near-fall event of the person.

The determining a label may be based on an inferred result by a local ML model, the local ML model being provided at the same site as the training data provider.

According to a second aspect, it is provided a training data provider for enabling training of a machine learning, ML, model, for monitoring a person based on a data feed capable of depicting a person. The training data provider comprises: a processor; and a memory storing instructions that, when executed by the processor, cause the training data provider to: obtain a data feed capable of depicting the person; select a level of anonymisation, from a plurality of levels of anonymisation; anonymise the data feed according to the selected level of anonymisation, resulting in a processed data feed; transmit the processed data feed as training data for training a central ML model in a central node; and receive an indication to increase or reduce the level of anonymisation from the central node; in which case said instructions are repeated, wherein the next iteration of the instructions to selecting is based on the indication to increase or reduce the level of anonymisation.

The instructions to anonymise may comprise instructions that, when executed by the processor, cause the training data provider to select another face of the same gender as the person in the data feed.

The training data provider may further comprise instructions that, when executed by the processor, cause the training data provider to: determine a label associated with the data feed; and include the label in association with the processed data feed.

The instructions to determine may comprise instructions that, when executed by the processor, cause the training data provider to determine the label is based on an inferred result by a local ML model, the local ML model being provided at the same site as the training data provider.

According to a third aspect, it is provided a computer program for enabling training of a machine learning, ML, model, for monitoring a person based on a data feed capable of depicting a person. The computer program comprises computer program code which, when executed on a training data provider causes the training data provider to: obtain a data feed capable of depicting the person; select a level of anonymisation, from a plurality levels of anonymisation; anonymise the data feed according to the selected level of anonymisation, resulting in a processed data feed; transmit the processed data feed as training data for training a central ML model in a central node; and receive an indication to increase or reduce the level of anonymisation from the central node; and repeat said computer program code, wherein the next iteration of the computer program code to select is based on the indication to increase or reduce the level of anonymisation.

Embodiments presented herein provide an improved way of anonymising a data feed for use as training data for an ML model. Specifically, one of a plurality levels of anonymisation is selected. In this way, the amount anonymisation can be tailored to the specific purpose, such that the anonymisation is not excessive to prevent training, while the anonymisation is as aggressive as possible to improve privacy of the person being depicted in the data feed.

<FIG> is a schematic diagram illustrating an environment in which embodiments presented herein can be applied. A person <NUM> to be monitored is at least part of the time present in a physical space <NUM>. The physical space <NUM> can e.g. be a room, a flat, a house, an office etc. A monitoring device <NUM> is configured to monitor the person <NUM> based on a sensor device <NUM> forming part of the monitoring device <NUM> or locally connected to the monitoring device <NUM>. The monitoring device <NUM> is also used for capturing data of the person <NUM> for purposes of training a local ML model <NUM> and a central ML model <NUM>. The sensor device <NUM> provides a data feed capable of depicting the person <NUM>, e.g. as a sequence of images (i.e. a video sequence). The sensor device <NUM> can be implemented as an infrared (IR) camera, a video camera, a lidar, a radar or any other suitable imaging technology. Additional sensor devices (not shown) can also be provided, providing respective data feeds. A training data provider <NUM> is used to provide training data for training the central ML model <NUM> as described in more detail below. The training data provider <NUM> can be provided as part of the monitoring device <NUM> or separately. In any case, the training data provider <NUM> is provided at the same site as the sensor device <NUM>. In this way, sensitive data feeds from the sensor device <NUM> are anonymised by the training data provider <NUM> to prevent privacy-sensitive sections of the data feed having to be communicated remotely.

The monitoring device <NUM> comprises the local ML model <NUM>. There may be one or more monitoring devices <NUM> working in parallel on the same or complementing scene. The monitoring device <NUM> can be connected to a network <NUM>, which can be an Internet protocol (IP) based network. The network <NUM> can e.g. comprise any one or more of a local wireless network, a cellular network, a wired local area network, a wide area network (such as the Internet), etc. Optionally, a central node <NUM>, containing the central ML model <NUM>, is also connected to the network <NUM>.

The local ML model <NUM> of the monitoring device <NUM> is used to predict current or future monitored states or events based on the data feed from the sensor device <NUM>. Specifically, the local ML model <NUM> is used to infer a result of monitoring states or events of the person <NUM> that can be used as labels in the training data. Non-limiting examples of monitoring states or events, all relating to the person, are: absent, present, lying in bed, lying on floor, breathing, near-fall event, fall event, distress, etc..

<FIG> is a flow chart illustrating embodiments of methods for enabling training of an ML model (e.g. the central ML model <NUM>), for monitoring a person based on a data feed capable of depicting a person.

In an obtain data feed step <NUM>, the training data provider <NUM> obtains a data feed capable of depicting the person. As explained above, the data feed can e.g. be based on data from one or more sensors such as infrared (IR) camera, a video camera, a lidar, a radar or any other suitable imaging technology. At this stage, the data feed has not been anonymised, and e.g. faces can potentially be seen in the data feed.

In a select level of anonymisation step <NUM>, the training data provider <NUM> selects a level of anonymisation, from a plurality of levels of anonymisation. The levels of anonymisation can e.g. include in, in order of increasing anonymisation: blurring of face, replacing face with a computer-generated face image, replacing face with a picture of someone else's face, blurring of entire body.

In an anonymise step <NUM>, the training data provider <NUM> anonymises the data feed according to the selected level of anonymisation, resulting in a processed data feed.

The anonymising can comprise, when a picture of someone else's face is used, selecting another face of the same gender as the person in the data feed. Alternatively or additionally, the face selection can be performed to achieve similar characteristics in terms as facial expression, which can be a valuable indicator to have in the training data. Alternatively or additionally, the face selection can be based on selecting another face of similar characteristics as the person in the data feed in terms of, hair colour, hair length, skin colour, etc..

Optionally, the original data feed (without anonymisation) is stored to allow training data with reduced anonymisation to be transmitted at a later stage if needed.

In an optional determine label step <NUM>, the training data provider <NUM> determines a label associated with the data feed. For instance, the label can indicate something that occurs relatively rarely, such as a near-fall event of the person. This type of event can be anonymised in the training data and still be valuable, since the training can e.g. be based on the movement characteristics of the body of the person, and may be determined without great dependence on facial expressions. Since such events happen rarely, any way that makes it possible to provide large amounts of data, such as provided by embodiments presented herein, is greatly valuable.

The determining of the label can e.g. be based on an inferred result by a local ML model, where the local ML model is provided at the same site as the training data provider <NUM>.

In an optional include label step <NUM>, the training data provider <NUM> includes the label (from step <NUM>) in association with the processed data feed.

In a transmit processed data step <NUM>, the training data provider <NUM> transmits the processed (i.e. anonymised and optionally labelled) data feed, to be used as training data for training a central ML model in a central node.

In an optional, receive adjustment indication step <NUM>, the training data provider <NUM> receives an indication to increase or reduce the level of anonymisation from the central node.

The method is then repeated, and when step <NUM> is performed, in the next iteration of step <NUM>, the selecting is based on the indication to increase or reduce the level of anonymisation, i.e. implementing a feedback loop. In this way, the level of optimisation is dynamically adjusted in accordance with the need of the central node.

Using embodiments presented herein, the level of anonymisation can be adjusted to achieve a balance between the level of detail required in the training and the impact on privacy for the person depicted in the data feed. In other words, the amount of privacy sensitive data forming part of the training data is reduced compared to if the training data should be useable for all types of ML model training. On the other hand, the level of detail provided in the training data is improved in cases where this is needed for successful training.

For instance, people counting, detecting absence/presence of people, or detecting near-fall events do not need a great amount of privacy-sensitive data, such as face data.

<FIG> is a schematic diagram illustrating components of the training data provider <NUM> of <FIG>. It is to be noted that, when the training data provider <NUM> is implemented in a host device such as the monitoring device <NUM>, one or more of the mentioned components can be shared with the host device. A processor <NUM> is provided using any combination of one or more of a suitable central processing unit (CPU), multiprocessor, microcontroller, digital signal processor (DSP), etc., capable of executing software instructions <NUM> stored in a memory <NUM>, which can thus be a computer program product. The processor <NUM> could alternatively be implemented using an application specific integrated circuit (ASIC), field programmable gate array (FPGA), etc. The processor <NUM> can be configured to execute the method described with reference to <FIG> above.

The training data provider <NUM> further comprises an I/O interface <NUM> for communicating with external and/or internal entities. For instance, the I/O interface <NUM> allows the training data provider <NUM> to communicate the network <NUM>. Optionally, the I/O interface <NUM> also includes a user interface.

Other components of the training data provider <NUM> are omitted in order not to obscure the concepts presented herein.

<FIG> shows one example of a computer program product <NUM> comprising computer readable means. On this computer readable means, a computer program <NUM> can be stored, which computer program can cause a processor to execute a method according to embodiments described herein. In this example, the computer program product is in the form of a removable solid-state memory, e.g. a Universal Serial Bus (USB) drive. As explained above, the computer program product could also be embodied in a memory of a device, such as the computer program product <NUM> of <FIG>. While the computer program <NUM> is here schematically shown as a section of the removable solid-state memory, the computer program can be stored in any way which is suitable for the computer program product, such as another type of removable solid-state memory, or an optical disc, such as a CD (compact disc), a DVD (digital versatile disc) or a Blu-Ray disc.

Claim 1:
A method for enabling training of a machine learning, ML, model, for monitoring a person based on a data feed capable of depicting a person, the method being performed by a training data provider (<NUM>) comprising a processor (<NUM>); and a memory (<NUM>) storing instructions to be executed by the processor (<NUM>), the method comprising:
obtaining (<NUM>) a data feed capable of depicting the person;
selecting (<NUM>) a level of anonymisation, from a plurality of levels of anonymisation;
anonymising (<NUM>) the data feed according to the selected level of anonymisation, resulting in a processed data feed;
transmitting (<NUM>) the processed data feed as training data for training a central ML model in a central node; and
receiving (<NUM>) an indication to increase or reduce the level of anonymisation from the central node;
and wherein the method is repeated, wherein the next iteration of the selecting (<NUM>) is based on the indication to increase or reduce the level of anonymisation.