Patent Description:
Motion data can be extracted from video data. Information, such as breathing data for a human subject, can be extracted from said motion data. However, there remains a need for alternative arrangements for generating and using motion data.

<CIT> describes a method for generating a breathing alert by capturing a video stream of a subject, generating a vector time series that includes a vector for each image frame of the video stream, estimating a breathing signal from the vector time series, determining one of large-scale motion and a breathing event of the subject based on the breathing signal, and. generating an alert if no breathing event is identified and no large-scale motion of the subject is identified within an event time interval.

In a first aspect there is described an apparatus according to claim <NUM>.

The movement data may comprise breathing pattern data, and/or pulse data.

The second plurality of pixels may comprise a random or pseudo-random selection of the pixels of the video image.

The apparatus may further comprise means for sensing said depth data.

The means for sensing said depth data may comprise a multi-modal camera including an infrared projector and an infrared sensor.

The means for processing the depth data may perform frequency domain filtering.

The frequency domain filtering may identify movements with frequencies in a normal range of breathing rates.

The apparatus may further comprise means for determining a mean distance measurement for the second plurality of pixels between their two successive instances.

The means for processing the determined depth data may generate said movement data based, at least in part, on determining whether a mean distance measurement between successive instances of depth data is greater than a threshold value.

The various means of the apparatus may comprise: at least one processor; and at least one memory including computer program code, the at least one memory and the computer program code configured, with the at least one processor, to cause the performance of the method of the second aspect, below.

In a second aspect, there is provided a computer-implemented method according to claim <NUM>.

The method may further comprise sensing said depth data.

Sensing said depth data may comprise using a multi-modal camera including an infrared projector and an infrared sensor.

Processing the depth data may comprise frequency domain filtering.

The method may further comprise determining a mean distance measurement for the second plurality of pixels between their two successive instances.

Processing the determined depth data may generate said movement data based, at least in part, on determining whether a mean distance measurement between successive instances of depth data is greater than a threshold value.

In a third aspect, there is provided a computer program according to claim <NUM>.

Examples will now be described, by way of non-limiting examples, with reference to the following schematic drawings, in which:.

Sleep disorders are associated with many problems, including psychiatric and medical problems. Sleep disorders are sometimes divided into a number of subcategories, such as intrinsic sleep disorders, extrinsic sleep disorders and circadian rhythm based sleep disorders.

Examples of intrinsic sleep disorders include idiopathic hypersomnia, narcolepsy, periodic limb movement disorder, restless legs syndrome, sleep apnoea and sleep state misperception.

Examples of extrinsic sleep disorders include alcohol-dependent sleep disorder, food allergy insomnia and inadequate sleep routine.

Examples of circadian rhythm sleep disorders include advanced sleep phase syndrome, delayed sleep phase syndrome, jetlag and shift worker sleep disorder.

Better understanding of sleep physiology and pathophysiology may aid in improving the care received by individuals suffering with such difficulties. Many sleep disorders are primarily diagnosed based on self-reported complaints. Lack of objective data can hinder case understanding and care provided. By way of example, data relating to breathing patterns of a patient may be valuable.

Non-contact, discrete longitudinal home monitoring may represent the most suitable option for monitoring sleep quality (including breathing patterns) over a period of time and may, for example, be preferred to clinic-based monitoring as the sleep patterns can vary between days depending on food intake, lifestyle and health status.

<FIG> is block diagram of a system, indicated generally by the reference numeral <NUM>, in accordance with an example embodiment. The system <NUM> includes a bed <NUM> and a depth camera <NUM>. The depth camera may, for example, be mounted on a tripod <NUM>. As discussed further below, the depth camera <NUM> may capture both depth data and video data of a scene including the bed <NUM> (and any patient on the bed).

In the use of the system <NUM>, a patient may sleep on the bed <NUM>, with the depth camera <NUM> being used to record aspects of sleep quality (such as breathing patterns indicated by patient movement). The bed <NUM> may, for example, be in a patient's home, thereby enabling home monitoring, potentially over an extended period of time.

<FIG> shows an example output, indicated generally by the reference numeral <NUM>, in accordance with an example embodiment. The output <NUM> is an example of depth data generated by the depth camera <NUM> of the system <NUM>. The depth data may be data of a scene that is fixed. For example, the scene may be captured using a camera having a fixed position and having unchanging direction and zoom. In this way, changes in the depth data can be monitored over time (e.g. over several hours of sleep) and may be compared with other data (e.g. collected over days, weeks or longer and/or collected from other users).

The depth camera <NUM> may produce depth matrices, where each element of the matrix corresponds to the distance from an object in the image to the depth camera. The depth matrices can be converted to grey images, as shown in <FIG> (e.g. the darker the pixel, the closer it is to the sensor). If the object is too close or too far away, the pixels may be set to zero such that they appear black.

<FIG> is a flow chart showing an algorithm, indicated generally by the reference numeral <NUM>, in accordance with an example embodiment.

The algorithm <NUM> starts at operation <NUM>, where depth data is received (for example from the camera <NUM> described above). The depth camera <NUM> (e.g. a camera incorporating an RGB-D sensor) may determine depth data for a first plurality of pixels of an image of a scene. The scene may relate to a human subject, for example captured during sleep.

Where the term "image" is used herein, it is to be understood that the image may be a video image, for example a series of frames of image data.

An example depth camera may capture per-pixel depth information via infra-red (IR) mediated structured light patterns with stereo sensing or time-of-flight sensing to generate a depth map. Some depth cameras include three-dimensional sensing capabilities allowing synchronized image streaming of depth information. Many other sensors are possible.

At operation <NUM>, a second plurality of pixels is selected. By way of example, <FIG> shows an example output, indicated generally by the reference numeral <NUM>, in accordance with an example embodiment. The output <NUM> shows the data of the output <NUM> and also shows the second plurality of pixels <NUM> (two of which are labelled in <FIG>). The second plurality of pixels may be randomly (or pseudo-randomly) selected from the plurality of pixels of the image <NUM> (although, as discussed further below, other distributions of the pixels <NUM> are possible). (Note that the pixels <NUM> are shown as relatively large pixels to ensure that they are visible in <FIG>. In reality, such pixels may be smaller.

As shown in <FIG>, the second plurality of pixels may be non-contiguous, for example distributed across the extent of the image of the scene. In this way, the second plurality of pixels can provide a plurality of "pinpricks" of data from the overall image. Clearly, the use of a plurality of pinprick-pixels can significantly reduce data storage and processing requirements when compared with considering the entire image. By "non-contiguous" it is meant that the second plurality of pixels are not located in such a way that they form a single contiguous area. In practice, the non-contiguous pixels may be located in a plurality of smaller contiguous groups that are not contiguous with one another. However, the best coverage of the scene may be obtained by ensuring the such groups are as small as possible, for example that each group is less than a threshold number of pixels. In some examples, the threshold number of pixels may be <NUM>, such that none of the second plurality of pixels is located immediately adjacent another.

The first plurality of pixels (obtain in the operation <NUM>) may include data from all pixels in the image <NUM>; however, this is not essential in all examples. Thus, the first plurality of pixels may be a subset of the full set of pixels of the image.

The second plurality of pixels comprises some or all of the first plurality of pixels. (In the event that the second plurality of pixels comprises all of the first plurality of pixels, the operation <NUM> may be omitted. ) Thus, whilst is some embodiments the second plurality of pixels is a subset of the first plurality of pixels, in other embodiments, the first and second plurality of pixels may be identical.

At operation <NUM>, depth data for successive instances of the second plurality of pixels are determined to generate data that can be used to provide an indication of a degree of movement between one instance of depth data and the next.

Finally, a movement indication, based on the outputs of the operation <NUM>, is provided in operation <NUM>. By way of example, the operation <NUM> may determine whether movement is detected in any of the second plurality of pixels. Alternatively, the movement indication may be generated from a combination (such as a sum, e.g. a weighted sum) of some or all of the second plurality of pixels.

The movement data generated in operation <NUM> is based on the depth data determined in operation <NUM>. Each instance of the second plurality of pixels comprises depth data of a consistent set of pixels of the scene such that the movement data is derived from data from the same set of pixels collected over time. In one example implementation, the successive instances of the of the second plurality of pixels were separated by <NUM> seconds (although, of course, other separations are possible). Separating successive instances of the depth data by a period such as <NUM> seconds even if more data are available may increase the likelihood of measurable movement occurring between instances and may reduce the data storage and processing requirements of the system.

More generally, an instance of one or more pixel may be the values of those pixels at a particular moment in time, for example in a single frame of a in a stream of image data such as a video. Successive instances of the pixel(s) may be adjacent frames of the stream. Alternatively, the video stream may be sampled at instances of time with successive instances of pixels being the values of pixels at each sample. Described above is an example where the sample period is used that is a fixed temporal value (e.g. a certain number of seconds); however, other approaches to sampling may be used. For example, the stream may be sampled (and an instance of the pixel(s) defined) in response to a repeating even that does not have a fixed period - for example each time a detected noise level associated with the monitored scene increases above a threshold amount, in which case the change in pixel data will be representative of movement between successive noises. Many possible approaches can be used to determine when an instance of the pixel data is defined.

As discussed further below, the movement measurements for each of the second plurality of pixels of the depth data may be determined on the basis of the mean distance evolution of the pixels. A distance measurement may be determined for the second plurality of pixels of each measurement instance, with the distance measurement enabling one measurement instance to be compared with other measurement instance(s).

The movement data generated in the operation <NUM> provides movement data for the second plurality of pixels. However, as discussed further below, movement in the second plurality of pixels may be representative of movement in the overall image. Moreover, by using only a subset of data points in the image (perhaps for a subset of time instance in the obtained data), the quantity of data stored and/or processed can be substantially reduced.

<FIG> is block diagram of a system, indicated generally by the reference numeral <NUM>, in accordance with an example embodiment. The system <NUM> comprises an imaging device <NUM> (such as the camera <NUM>), a data storage module <NUM>, a data processing module <NUM> and a control module <NUM>.

The data storage module <NUM> may store measurement data (e.g. the depth data for the first plurality of pixels, as obtained in the operation <NUM> of the algorithm <NUM>) under the control of the control module <NUM>, with the stored data being processed by data processor <NUM> to generate an output (e.g. the output of the operation <NUM> of the operation <NUM>). The output may be provided to the control module <NUM>. In an alternative arrangement, the data storage module <NUM> may store the depth data of the second plurality of depth data (e.g. outputs of the operation <NUM>), with the stored data being processed by data processor <NUM> to generate the output (e.g. the output of the operation <NUM>). A potential advantage of storing the second plurality of data is a reduced data storage requirement.

The camera <NUM> of the system <NUM> and/or the imaging device <NUM> of the system <NUM> may be a multimodal camera comprising a colour (RGB) camera and an infrared (IR) projector and infrared sensor. The sensor may send an array of near infra-red (IR) light into the field-of-view of the camera <NUM> or the imaging device <NUM>, with a detector receiving reflected IR and an image sensor (e.g. a CMOS image sensor) running computational algorithms to construct a real-time, three-dimensional depth value mesh-based video. Information obtained and processed in this way may be used, for example, to identify individuals, their movements, gestures and body properties and/or may be used to measure size, volume and/or to classify objects. Images from depth cameras can also be used for obstacle detection by locating a floor and walls.

In one example implementation, the imaging device <NUM> was implemented using a Kinect (RTM) camera provided by the Microsoft Corporation. In one example, the frame rate of the imaging device <NUM> was <NUM> frames per second.

Depth frames may be captured by the imaging device <NUM> and stored in the data storage module <NUM> (e.g. in the form of binary files). In an example implementation, the data storage <NUM> was used to record depth frames for subjects sleeping for period ranging between <NUM> and <NUM> hours. In one example implementation, the data storage requirement for one night of sleep was of the order of 200GB when the entirety of each frame was stored; however, it was possible to reduce this storage requirement by approximately <NUM>% using the approach described herein.

The stored data can be processed by the data processor <NUM>. The data processing may be online (e.g. during data collection), offline (e.g. after data collection) or a combination of the two. The data processor <NUM> may provide an output, as discussed further below.

<FIG> is a flow chart showing an algorithm, indicated generally by the reference numeral <NUM>, showing an example implementation of the operation <NUM> of the algorithm <NUM>, in accordance with an example embodiment. As described above, the operation <NUM> determines movement between successive instances of the captured depth data.

The algorithm <NUM> starts at operation <NUM> where a mean distance value for the second plurality of pixels of an image is determined. Then, at operation <NUM>, distances between images instances (based on the distances determined in operation <NUM>) are determined. As discussed further below, a distance between images instances that is higher than a threshold value may be indicative of movement between successive images.

By way of example, consider the depth data of the second plurality of pixels generated in the operation <NUM>.

After removing all the zero pixels in the frames the mean distance of the second plurality of pixels at instance i is defined by: <MAT> Where:.

In this context, the "distance" of a pixel refers to the difference in its depth between the two instances. For example, if the pixel has a depth of <NUM> in a first instance and a depth of <NUM> in a second instance then the distance of the pixel between the two instances is <NUM> - <NUM> = <NUM>.

Having computed the mean distance of the second plurality of pixels of successive frames of a dataset (the operation <NUM>), a determination of distance between image instances (indicative of movement in the relevant scene) can be made (the operation <NUM>).

One method for determining movement is as follows. First, a mean is defined based on the set of all the non-zero value pixels. Next, if the difference between the mean distance of two consecutive frames is above a certain threshold θ<NUM>, a change in position is noticed. This is given by: M(i) - M(i - <NUM>) ≥ θ<NUM>.

Several values based on standard deviations and maximum differences are tested in order to find the best threshold values (as discussed further below).

<FIG> is a flow chart showing an algorithm, indicated generally by the reference numeral <NUM>, in accordance with an example embodiment. The algorithm <NUM> starts at operation <NUM> where an initialisation phase is carried out. The initialisation phase is conducted without a patient being present (such that there should be no movement detected). As described below, the initialisation phase is used to determine a noise level in the data. Next, at operation <NUM>, a data collection phase is carried out. Finally, at operation <NUM>, instances of the data collected in operation <NUM> having data indicative of movement are identified (thereby implementing the operation <NUM> of the algorithm <NUM> described above). The instances identified in operation <NUM> are based on a threshold distance level that is set depending on the noise level determined in the operation <NUM>.

<FIG> shows an initialisation data, indicated generally by the reference numeral <NUM> in accordance with an example embodiment. The initialisation data <NUM> includes mean distance data <NUM>, a first image <NUM>, a second image <NUM> and a representation of the noise <NUM>. The first image <NUM> corresponds to the mean distance data of a data point <NUM> and the second image <NUM> corresponds to the mean distance data of a data point <NUM>.

It should be noted that although the data <NUM> may be based on full images (similar to the image <NUM> discussed above), similar initialisation data could have been generated based on the second plurality of pixels described above.

The mean distance data <NUM> shows how the determined mean frame distance changes over time (a time period of <NUM> minutes is shown in <FIG>). As the data <NUM> was collected without motion (e.g. without a patient being on the bed <NUM> of the system <NUM>), the variations in the mean distance data <NUM> are representative of noise. The noise representation <NUM> expresses the noise as a Gaussian distribution. That distribution can be used to determine a standard deviation for the noise data (as indicated by the standard deviation shown in the data <NUM>). A maximum difference between two consecutive mean values is also plotted on the output <NUM>.

The operation <NUM> seeks to assess the intrinsic noise in the frames of the output <NUM>. In this way, the intrinsic variation in the data at rest can be determined so that the detection of motion will not be confused by noise.

The operation <NUM> may be implemented using the principles of the algorithm <NUM> described above. Thus, video data (i.e. the output <NUM>) may be received. Distances in the selected second plurality of pixels can then be determined (see operation <NUM> of the algorithm <NUM>). (Although, as noted above, the noise could be based on the entire image, e.g. the first plurality of pixels.

In the data collection phase <NUM> of the algorithm <NUM>, the subject goes to sleep and the relevant dataset is collected. Then, in operation <NUM>, the knowledge of the noise is exploited to determine the likelihood that a change in distance data is indicative of actual movement (rather than noise). By way of example, movement may be deemed to have occurred in the event that determined distance data is more than <NUM> standard deviations (as determined in operation <NUM>) away from the previous mean distance.

An experiment has been conducted on fourteen nights including eleven different subjects in order to validate the reproducibility of the experiment. The data were stored as videos which serve as ground truth. The true detection of movements was manually noted looking at the videos and was compared to the detection of movements provided by the algorithm being tested.

<FIG> show results, indicated generally by the reference numerals <NUM> and <NUM> respectively, in accordance with example embodiments. The results are based on a mean distance evolution (i.e. how the mean distance determinations change over time) for the fourteen subjects. The percent of good detection (true positives) averaging all experiments is shown together with the overestimation of movements (false positives). The results <NUM> and <NUM> are based on frames of image data having the form described above with reference to <FIG> (i.e. without selecting a subset of the pixels for processing).

The results <NUM> show the number of correct movement identifications (true positives) and the number of false movement identification (false positives) for different standard deviation threshold levels. With the movement threshold at two standard deviations (such that a mean distance change of at least two standard deviations is detected), a true positives measurement of <NUM>% was detected and a false positives measurements of <NUM>% was detected. As the threshold level was increased, the number of false positives reduced (to zero at five standard deviations). However, at five standard deviations, the true positives level had reduced to <NUM>%. Accordingly, the results <NUM> show poor performance.

The results <NUM> show the number of correct movement identifications (true positives) and the number of false movement identification (false positives) when using maximum distance differences as the threshold value. The performance was also poor.

<FIG> show results, indicated generally by the reference numerals <NUM> and <NUM> respectively, in accordance with example embodiments. The results are based on a mean distance evolution for the fourteen subjects. The percent of good detection (true positives) averaging all experiments is shown together with the overestimation of movements (false positives). The results <NUM> and <NUM> are based on considering pixel depth evolution of a plurality of pixels of image data (e.g. the second plurality of pixels discussed above).

The results <NUM> show the number of correct movement identifications (true positives) and the number of false movement identification (false positives) for different standard deviation threshold levels. With the movement threshold at <NUM> standard deviations, a true positives measurement of <NUM>% was detected and a false positives measurement of <NUM>% was detected. Thus, the results for the second plurality of pixels were significantly better than the results for the full frame arrangement described above with reference to <FIG>. The results <NUM> show a similarly good performance when using maximum distance differences as the threshold value.

The algorithm <NUM> starts at operation <NUM> where measurement data is obtained by sensing. The operation <NUM> may, for example, implement some or all of the algorithms <NUM> or <NUM> described above. The operation <NUM> may, for example, provide data indicative of movement over time.

At operation <NUM>, frequency-domain filtering of the measurement data is conducted. Thus, the operation <NUM> may implement means for processing determined changes in depth data by incorporating frequency domain filtering. Such filtering may, for example, be used to identify movement with frequencies within a particular range.

One example use of the principles described herein is in the monitoring of breathing patterns of a patient. In such an embodiment, the operation <NUM> may perform frequency-domain filtering in order to identify movements with frequencies in a normal range of breathing rates. Thus, movement data can be filtered to identify movement that might be indicative of breathing. For example, a band-pass filter centred on typical breathing frequencies may be used to implement the operation <NUM>.

<FIG> shows output data, indicated generally by the reference numeral <NUM>, in accordance with an example embodiment. The data <NUM> compares the performance of depth data processing in accordance with the principles described herein (labelled "Kinect" in <FIG>), with the performance of a measurement belt (labelled "Belt" in <FIG>).

The output <NUM> shows the "Kinect" and "Belt" performance in the time domain, wherein movement amplitude is plotted against time. The output <NUM> shows the "Kinect" performance in the frequency domain. The output <NUM> shows the "Belt" performance in the frequency domain.

As can be seen in <FIG>, the performance of depth data processing in accordance with an example embodiment described herein ("Kinect") is similar to the performance of with an example measurement belt ("Belt").

In the arrangement described above with reference to <FIG>, the selection of pixels (in operation <NUM> of the algorithm <NUM>) was random (or pseudorandom). In other embodiments, the selection of pixels may be subject to a distribution function (with or without a random or pseudorandom element). A variety of functions are described herein (which functions may be used alone or in any combination).

<FIG> shows an example image, indicated generally by the reference numeral <NUM>, being processed in accordance with an example embodiment. A plurality of pixels <NUM> that collectively form the pixels selected in the operation <NUM> are shown. The pixels are spread according to a function that favours locations close to the centre of the image. The distribution <NUM> also has a random element. The logic here is that the camera/imaging device is likely to be directed such that objects of interest are close to the centre of the field of view.

The weighting of a distribution function may be at least partially dependent on visual characteristics of the scene. For example, areas above the level of the bed <NUM> in the system <NUM> described above may be weighted higher in a distribution function than areas with surfaces below the surface on the bed (on the assumption that movement is more likely to occur above the level of the bed than below the level of the bed). Alternatively, or in addition, areas on the bed <NUM> in the system <NUM> may be weighted higher in the distribution function. The second plurality of pixels may be selected on the basis of such distribution functions, favouring pixel locations within areas that have a higher weighting.

<FIG> shows an example image, indicated generally by the reference numeral <NUM>, being processed in accordance with an example embodiment. A plurality of pixels <NUM> that collectively form the pixels selected in the operation <NUM> are shown. The pixels are spread randomly (or pseudorandomly) but subject to being positioned on a bed identified in the image. The logic here is that a sleeping patient can be expected to be located on the bed.

The examples described above generally relate to sleep monitoring in general, and the detection of data relating to sleep disorders in particular. This is not essential to all embodiments. For example, the principles discussed herein can be used to determine movement in a scene for other purposes. For example, a similar approach may be taken to detect movement in a largely static scene, for example for monitoring an area to detect the movement of people, animals, vehicles, etc. within it.

Moreover, the examples described above generally relate to determining breathing patterns of a patient. This is not essential to all embodiments. Many other movements may be detected. For example, the principles described herein could be applied to the detection of a heartbeat of a patient.

For completeness, <FIG> is a schematic diagram of components of one or more of the example embodiments described previously, which hereafter are referred to generically as processing systems <NUM>. A processing system <NUM> may have a processor <NUM>, a memory <NUM> closely coupled to the processor and comprised of a RAM <NUM> and ROM <NUM>, and, optionally, user input <NUM> and a display <NUM>. The processing system <NUM> may comprise one or more network/apparatus interfaces <NUM> for connection to a network/apparatus, e.g. a modem which may be wired or wireless. Interface <NUM> may also operate as a connection to other apparatus such as device/apparatus which is not network side apparatus. Thus direct connection between devices/apparatus without network participation is possible. User input <NUM> and display <NUM> may be connected to a remote processor like ground control station. Remote connection may be LTE or <NUM> type fast connection between remote processor and processor.

The memory <NUM> may comprise a non-volatile memory, such as a hard disk drive (HDD) or a solid state drive (SSD). The ROM <NUM> of the memory <NUM> stores, amongst other things, an operating system <NUM> and may store software applications <NUM>. The RAM <NUM> of the memory <NUM> is used by the processor <NUM> for the temporary storage of data. The operating system <NUM> may contain code which, when executed by the processor implements aspects of the algorithms <NUM>, <NUM>, <NUM> and <NUM> described above. Note that in the case of small device/apparatus the memory can be most suitable for small size usage i.e. not always hard disk drive (HDD) or solid state drive (SSD) is used.

The processor <NUM> may take any suitable form. For instance, it may be a microcontroller, a plurality of microcontrollers, a processor, or a plurality of processors.

The processing system <NUM> may be a standalone computer, a server, a console, or a network thereof. The processing system <NUM> and needed structural parts may be all inside device/apparatus such as IoT device/apparatus i.e. embedded to very small size.

In some example embodiments, the processing system <NUM> may also be associated with external software applications. These may be applications stored on a remote server device/apparatus and may run partly or exclusively on the remote server device/apparatus. These applications may be termed cloud-hosted applications. The processing system <NUM> may be in communication with the remote server device/apparatus in order to utilize the software application stored there.

<FIG> show tangible media, respectively a removable memory unit <NUM> and a compact disc (CD) <NUM>, storing computer-readable code which when run by a computer may perform methods according to example embodiments described above. The removable memory unit <NUM> may be a memory stick, e.g. a USB memory stick, having internal memory <NUM> storing the computer-readable code. The memory <NUM> may be accessed by a computer system via a connector <NUM>. The CD <NUM> may be a CD-ROM or a DVD or similar. Other forms of tangible storage media may be used. Tangible media can be any device/apparatus capable of storing data/information which data/information can be exchanged between devices/apparatus/network.

Reference to, where relevant, "computer-readable storage medium", "computer program product", "tangibly embodied computer program" etc., or a "processor" or "processing circuitry" etc. should be understood to encompass not only computers having differing architectures such as single/multi-processor architectures and sequencers/parallel architectures, but also specialised circuits such as field programmable gate arrays FPGA, application specify circuits ASIC, signal processing devices/apparatus and other devices/apparatus. References to computer program, instructions, code etc. should be understood to express software for a programmable processor firmware such as the programmable content of a hardware device/apparatus as instructions for a processor or configured or configuration settings for a fixed function device/apparatus, gate array, programmable logic device/apparatus, etc..

As used in this application, the term "circuitry" refers to all of the following: (a) hardware-only circuit implementations (such as implementations in only analogue and/or digital circuitry) and (b) to combinations of circuits and software (and/or firmware), such as (as applicable): (i) to a combination of processor(s) or (ii) to portions of processor(s)/software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a server, to perform various functions) and (c) to circuits, such as a microprocessor(s) or a portion of a microprocessor(s), that require software or firmware for operation, even if the software or firmware is not physically present.

Similarly, it will also be appreciated that the flow diagrams of <FIG>, <FIG>, <FIG> and <FIG> are examples only and that various operations depicted therein may be omitted, reordered and/or combined.

Other variations and modifications will be apparent to persons skilled in the art upon reading the present specification. For example, it would be possible to extend the principles described herein to other applications, such as the control of robots or similar objects.

Moreover, the disclosure of the present application should be understood to include any novel features or any novel combination of features either explicitly or implicitly disclosed herein or any generalization thereof and during the prosecution of the present application or of any application derived therefrom, new claims may be formulated to cover any such features and/or combination of such features.

Claim 1:
An apparatus (<NUM>) comprising:
means for receiving (<NUM>) depth data (<NUM>) for a first plurality of pixels of a video image of a scene; characterized by
means for selecting (<NUM>) a second plurality of pixels such that a distribution of said second plurality of pixels complies with a distribution function, wherein the second plurality of pixels is a subset of the first plurality of pixels and wherein the distribution function has a weighting that is at least partially dependent on location of a respective pixel within the scene, and wherein the second plurality of pixels are non-contiguous such that the second plurality of pixels are located in a plurality of contiguous groups that are not contiguous with one another, and each contiguous group comprises less than a threshold number of pixels;
means for determining (<NUM>) depth data over time for each of the second plurality of pixels (<NUM>) of the video image; and
means for processing the determined depth data of successive instances of the second plurality of pixels to generate (<NUM>) movement data providing an indication of a degree of movement between one instance of depth data and the next, wherein each instance of the second plurality of pixels comprises depth data of the second plurality of pixels.