Patent Description:
LiDAR can be used to measure or track objects. For example, a user device such as a phone, that is equipped with LiDAR, can detect objects such as people in the region around the user device. LiDAR can be used to monitor the relative movement between the objects and the user device, to identify the type of objects or for any other suitable purpose. The resolution of such LiDAR devices can be limited by the LiDAR hardware within the device. This can limit the accuracy of the LiDAR and restrict the applications for which the LiDAR can be used.

<CIT> (D1) relates to a LIDAR-only lock-on tracking system that uses multiple beams arranged in a pattern, or a signal-of-interest (SOI) array, such that the beams do not move appreciably relative to the target during data gathering.

<CIT> (D2) relates to a method of accurately projecting a laser pattern on a target or work surface and continuously compensating for relative dynamic movement between the laser projector and the target's surface.

According to various, but not necessarily all, examples of the disclosure there is provided an apparatus comprising means for:.

The means may be for identifying the at least one areas of interest of the at least one object.

The means may be for using images obtained by an imaging device to predict the location of the at least one area of interest of the at least one object.

The means may be for using information obtained from the at least one LiDAR device to predict the location of the at least one area of interest of the at least one object.

The at least one object may comprise at least part of a person and the location of the area of interest of at least one object is predicted using predictions based on human behaviour.

The location of the at least one area of interest relative to the at least one LiDAR device may be estimated using information obtained from one or more sensors configured to measure movement of the at least one LiDAR device.

The means may be for estimating the delay to be added to the transmission of the measurement location projection pattern by calculating one or more expected positions of the at least one area of interest based on the relative velocity between the at least one object and the LiDAR device, calculating a spatial shift for the measurement location projection pattern based on this relative velocity and the current and desired projected measurement locations, and calculating a corresponding time delay to produce the spatial shift.

The apparatus may comprise means for causing movement of at least part of the at least one LiDAR device to provide relative movement between the at least one object and the at least one LiDAR device.

The means may be for detecting relative movement between a plurality of objects and the at least one LiDAR device and predicting a location of at least one area of interest for at least some of the plurality of objects and adding a delay to transmission of the measurement location projection pattern to optimise the incidence of measurement locations from the measurement location projection pattern on the areas of interest for the at least some of the plurality of objects.

The means may be for interpolating between consecutive measurements detected by the at least one LiDAR device to account for the delay added to the transmission of the measurement location projection pattern.

The measurement location projection pattern may comprise an array of evenly spaced measurement locations.

According to various, but not necessarily all, examples of the disclosure there is provided an apparatus comprising at least one processor; and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform:.

An electronic device comprising an apparatus as described herein.

A communications device comprising an apparatus as described herein.

A vehicle comprising an apparatus as described herein.

According to various, but not necessarily all, examples of the disclosure there is provided a method comprising:.

According to various, but not necessarily all, examples of the disclosure there is provided a computer program comprising computer program instructions that, when executed by processing circuitry, cause:.

LiDAR devices can be provided in user devices such as mobile phones or in other types of devices such as vehicles. The LiDAR devices can be used to identify nearby objects and/or to determine the relative positions of the objects. In some examples the LiDAR devices can be used to track the relative positions of the objects and the LiDAR devices. The resolution of the LiDAR devices is limited by the hardware of the LiDAR device and the device comprising the LiDAR device. Examples of the disclosure provide improved resolution of the LiDAR by adding a delay to the transmission of a measurement location projection pattern of the LiDAR device.

<FIG> shows an example device <NUM> that could be used to implement examples of the disclosure. The device <NUM> comprises a LiDAR device <NUM>, an apparatus <NUM> and an imaging device <NUM>. The device <NUM> could be a user device such as a mobile phone or other personal communication device. In some examples the device <NUM> could be provided as part of a vehicle or other autonomous device.

In the example shown in <FIG> the LiDAR device <NUM> is being used to measure an object <NUM>. In this example the object <NUM> is a person. Other types of objects could be measured in other examples of the disclosure.

The LiDAR device <NUM> can comprise any means that can be configured to use LiDAR signals to measure one or more objects <NUM>. The LiDAR device <NUM> can be used to detect the distance and position of one or more objects <NUM>, to identify one or more objects <NUM> or for any other suitable purpose.

The LiDAR device <NUM> can be configured to measure areas of interest of the object <NUM>. This can help to identify the objects <NUM>, and/or can enable the objects <NUM> to be identified more accurately. The areas of interest can comprise an area of the object <NUM> in which there are more variations or complex information compared to other areas of the object <NUM>. For example, In <FIG> the object <NUM> comprises the face of a person. The area around the nose has a larger variation in depth compared to the cheeks. This can enable more information to be obtained from the area around the nose than from the cheeks. In such examples the areas of interest would comprise the areas around the nose.

In some examples the area of interest of an object <NUM> could comprise part of the object <NUM>. In other examples the area of interest could comprise the whole of the object <NUM> or substantially the whole of the object <NUM>.

The LiDAR device <NUM> is configured to transmit pulses of light from a laser or other suitable light source. The reflections of the light from the one or more objects <NUM> can be detected and used to measure or provide information about the one or more objects <NUM>.

The LiDAR device <NUM> can be configured to transmit a measurement location projection pattern <NUM>. The measurement location projection pattern <NUM> can comprise a plurality of measurement locations that are transmitted by the LiDAR device <NUM>. The measurement locations can be one or more areas of light. The measurement locations can be areas of light transmitted from a laser or any other suitable light source.

The areas of light within the measurement location projection pattern <NUM> can have any suitable shape. In some examples the areas of light can comprise a plurality of dots and the dots can be provided in an array to provide a dot projection pattern.

The areas of light within the measurement location projection pattern <NUM> can be arranged in any suitable pattern. In some examples the areas of light can be arranged in a regular array comprising a series of regularly spaced columns and rows. Other types of arrays could be used in other examples of the disclosure, for example the arrays could comprise a sequence of concentric circles or any other suitable geometric patterns. The measurement location projection pattern <NUM> can comprise an array of evenly spaced, or substantially evenly spaced, measurement locations.

When the measurement location projection pattern <NUM> is incident on an object <NUM> the light is reflected back towards the LiDAR device <NUM> or any other suitable detection means. The LiDAR device <NUM>, or any other suitable detection means, can detect the reflected light and use the timings of the reception of the reflected light signals to measure the location of the object <NUM>. This information can be used to identify the object <NUM> or to distinguish one type of object from another type of object. The identification of the objects <NUM> could be via the application of an object detection algorithm on the geometries detected and mapped by data obtained by the LiDAR device <NUM>.

The apparatus <NUM> that is provided within the device <NUM> can comprise a controller <NUM> comprising a processor <NUM> and memory <NUM> that can be as shown in <FIG>. The apparatus <NUM> can be configured to enable control of the device <NUM>. For example, the apparatus <NUM> can be configured to control the LiDAR device <NUM> to enable measurement of one or more objects. The apparatus <NUM> can also be configured to control the images that are captured by the imaging device <NUM> and/or to control any other functions that could be implemented by the device <NUM>.

The imaging device <NUM> can comprise any means for capturing an image. The imaging device <NUM> can comprise one or more cameras or any other suitable means for capturing images. In the example of <FIG> only one imaging device <NUM> is shown. However, it is to be appreciated that the device <NUM> could comprise a plurality of imaging devices <NUM> that could be positioned in different locations within the device <NUM>. For example, the device <NUM> could comprise a front facing imaging device <NUM> and a rear facing imaging device <NUM>.

The imaging device <NUM> can comprise one or more sensors where the sensors can be configured to detect images. The sensors of the imaging device <NUM> can be coupled to the apparatus <NUM> to enable detected images to be stored in an image storage module. In some examples the image storage module could be part of the memory <NUM> of the apparatus <NUM>. The sensors of the imaging device <NUM> can comprise any suitable type of image sensor. For instance, the sensor of the imaging device <NUM> can comprise a digital image sensor such as a charge-coupled-device (CCD) or a complementary metal-oxide-semiconductor (CMOS).

The imaging device <NUM> can be controlled by the apparatus <NUM> to enable images to be captured. Once an image has been captured it can be stored in the image storage module and/or transmitted to a third party. The images that are captured by the imaging device <NUM> can be used to identify objects <NUM> within an image and/or to enable tracking of the objects <NUM>. The tracking of the objects <NUM> comprises determining how the position of the object <NUM> changes over a period of time. In some examples the tracking of the object <NUM> can comprise determining a trajectory of the object <NUM>.

Only components of the device <NUM> that are referred to in the following description are shown in <FIG>. It is to be appreciated that the device <NUM> could comprise additional components that are not shown in <FIG>. For instance, the device <NUM> could comprise a power source, one or more transceivers and/or any other suitable components.

In some examples the device <NUM> could comprise means for causing motion of the LiDAR device <NUM>. The means for causing motion of the LiDAR device <NUM> could be configured to cause motion of just the LiDAR device <NUM> and/or to cause motion of the whole of the device <NUM>. In some examples the means for causing motion of the LiDAR device <NUM> could comprise one or more motors that can be configured to cause a vibration of the LiDAR device <NUM> and/or the device <NUM>. This vibration can create relative movement between the LiDAR device <NUM> and the object <NUM> that is to be measured by the LiDAR device <NUM>.

<FIG> shows an example apparatus <NUM>. The apparatus <NUM> illustrated in <FIG> can be a chip or a chip-set. The apparatus <NUM> can be provided within a device <NUM> such as a mobile phone, personal electronics device or any other suitable type of device <NUM>. In some examples the apparatus <NUM> could be provided within a vehicle or other device that monitors the objects <NUM> within the surroundings. The apparatus <NUM> could be provided within devices <NUM> as shown in <FIG>.

In the example of <FIG> the apparatus <NUM> comprises a controller <NUM>. In the example of <FIG> the implementation of the controller <NUM> can be as controller circuitry. In some examples the controller <NUM> can be implemented in hardware alone, have certain aspects in software including firmware alone or can be a combination of hardware and software (including firmware).

As illustrated in <FIG> the controller <NUM> can be implemented using instructions that enable hardware functionality, for example, by using executable instructions of a computer program <NUM> in a general-purpose or special-purpose processor <NUM> that can be stored on a computer readable storage medium (disk, memory etc.) to be executed by such a processor <NUM>.

The processor <NUM> can also comprise an output interface via which data and/or commands are output by the processor <NUM> and an input interface via which data and/or commands are input to the processor <NUM>.

The memory <NUM> is configured to store a computer program <NUM> comprising computer program instructions (computer program code <NUM>) that controls the operation of the apparatus <NUM> when loaded into the processor <NUM>. The computer program instructions, of the computer program <NUM>, provide the logic and routines that enable the apparatus <NUM> to perform the methods illustrated in <FIG> and <FIG>. The processor <NUM> by reading the memory <NUM> is able to load and execute the computer program <NUM>.

The apparatus <NUM> therefore comprises: at least one processor <NUM>; and at least one memory <NUM> including computer program code <NUM>, the at least one memory <NUM> and the computer program code <NUM> configured to, with the at least one processor <NUM>, cause the apparatus <NUM> at least to perform:.

As illustrated in <FIG> the computer program <NUM> can arrive at the apparatus <NUM> via any suitable delivery mechanism <NUM>. The delivery mechanism <NUM> can be, for example, a machine readable medium, a computer-readable medium, a non-transitory computer-readable storage medium, a computer program product, a memory device, a record medium such as a Compact Disc Read-Only Memory (CD-ROM) or a Digital Versatile Disc (DVD) or a solid-state memory, an article of manufacture that comprises or tangibly embodies the computer program <NUM>. The delivery mechanism can be a signal configured to reliably transfer the computer program <NUM>. The apparatus <NUM> can propagate or transmit the computer program <NUM> as a computer data signal. In some examples the computer program <NUM> can be transmitted to the apparatus <NUM> using a wireless protocol such as Bluetooth, Bluetooth Low Energy, Bluetooth Smart, 6LoWPan (IPv<NUM> over low power personal area networks) ZigBee, ANT+, near field communication (NFC), Radio frequency identification, wireless local area network (wireless LAN) or any other suitable protocol.

The computer program <NUM> comprises computer program instructions for causing an apparatus <NUM> to perform at least the following:.

The computer program instructions can be comprised in a computer program <NUM>, a non-transitory computer readable medium, a computer program product, a machine readable medium. In some but not necessarily all examples, the computer program instructions can be distributed over more than one computer program <NUM>.

Although the memory <NUM> is illustrated as a single component/circuitry it can be implemented as one or more separate components/circuitry some or all of which can be integrated/removable and/or can provide permanent/semi-permanent/ dynamic/cached storage.

Although the processor <NUM> is illustrated as a single component/circuitry it can be implemented as one or more separate components/circuitry some or all of which can be integrated/removable. The processor <NUM> can be a single core or multi-core processor.

References to "computer-readable storage medium", "computer program product", "tangibly embodied computer program" etc. or a "controller", "computer", "processor" etc. should be understood to encompass not only computers having different architectures such as single /multi- processor architectures and sequential (Von Neumann)/parallel architectures but also specialized circuits such as field-programmable gate arrays (FPGA), application specific circuits (ASIC), signal processing devices and other processing circuitry.

As used in this application, the term "circuitry" can refer to one or more or all of the following:.

The blocks illustrated in <FIG> and <FIG> can represent steps in a method and/or sections of code in the computer program <NUM>. The illustration of a particular order to the blocks does not necessarily imply that there is a required or preferred order for the blocks and the order and arrangement of the block can be varied. Furthermore, it can be possible for some blocks to be omitted.

<FIG> shows an example method according to examples of the disclosure. The method could be implemented using an apparatus <NUM> and/or device <NUM> as described above or using any other suitable type of device or apparatus.

At block <NUM> the method comprises detecting relative movement between at least one object <NUM> and the LiDAR device <NUM>. The relative movement can be caused by movement of the LiDAR device <NUM>, movement of the device <NUM> that the LiDAR device <NUM> is comprised within, movement of the object <NUM> or any combination of these.

The LiDAR device <NUM> is configured to provide a measurement location projection pattern <NUM>. The measurement location projection pattern <NUM> comprises a plurality of measurement locations that are transmitted by the LiDAR device <NUM>. The measurement locations within the measurement location projection pattern <NUM> comprise areas of light. The areas of light can be arranged in a regular array or in any other suitable configurations.

The LiDAR device <NUM> can be configured to transmit the measurement location projection pattern <NUM> at predetermined intervals. The LiDAR device <NUM> can have a predetermined rate at which the measurement location projection pattern <NUM> is transmitted. The rate at which the measurement location projection pattern <NUM> is transmitted can be controlled by the apparatus <NUM> or by any other suitable means. The apparatus <NUM> can be configured to change the intervals between the transmission of successive measurement location projection patterns <NUM> in accordance with examples of the disclosure. For example, the apparatus <NUM> can be configured to introduce a delay into the transmission of a measurement location projection pattern <NUM>. The delay can be added to the transmission of the measurement location projection pattern <NUM> by controlling power to the LiDAR device <NUM>. In such examples the LiDAR device <NUM>, or components of the LiDAR device can be turned on and off as determined by the delay.

Any suitable means can be used to detect the relative movement between the at least one object <NUM> and the LiDAR device <NUM>. For instance, the movement can be detected by using any one or more of: images from the imaging device <NUM>, information obtained from the LiDAR device <NUM>, one or more motion sensors or any other suitable means.

At block <NUM> the method comprises predicting a location of at least one area of interest of the at least one object <NUM> relative to the at least one LiDAR device <NUM> at a given time.

The areas of interest can comprise areas of the object <NUM> in which there are more variations or information compared to other areas of the object <NUM>. The areas of interest can comprise the areas of the object <NUM> that have the highest levels of complexity. The level of complexity can be related to the level of variation of depth within a given area. For example, if the object <NUM> comprises the face of a person, the area around the nose has a larger variation in depth compared to the cheeks. These areas are more complex and can comprise greater variations in depth. In such examples the areas of interest would comprise the areas around the nose.

In some examples the apparatus <NUM> can be configured to identify the at least one areas of interest of the at least one object <NUM>. The apparatus <NUM> can be configured to identify the areas of the object <NUM> or the plurality of objects that provide the useful information. In some examples the apparatus <NUM> can also identify the areas of the object <NUM> that do not provide useful information or do not provide as much useful information. In some examples the apparatus <NUM> can be configured to identify the areas that are of interest and also the areas that are not of interest.

Any suitable means can be used to identify the one or more areas of interest. In some examples the apparatus <NUM> can be configured to obtain images from the imaging device <NUM>. The apparatus <NUM> could then analyse these images, using pattern recognition, machine learning, computer vision techniques or any other suitable technique, to identify the one or more areas of interest. Other suitable means could be used to identify the areas of interest in other examples of the disclosure.

Once the one or more areas of interest of the object <NUM> is known and the movement of the object <NUM> has been detected any suitable means can be used to predict the location of the area of interest of the object <NUM> at a given time. The given time can be a time that is a short time interval ahead of the current time. For example, the given time could be a time that occurs in less than a second or in one or more second's time (for example, in several seconds' time).

Any suitable means, or combinations or means, can be used to predict the location of the areas of interest of the object <NUM>.

In some examples the apparatus <NUM> can be configured to use images obtained by the imaging device <NUM> to predict the locations of areas of interest of the object <NUM> at the given point in time. For example, a plurality of images obtained by the imaging device <NUM> could be used to plot a trajectory of an object <NUM> and from that trajectory predict where the object <NUM> will be in a future point in time.

In some examples the images obtained by the imaging device <NUM> could indicate the location of other objects within the region around the object. This information could also be used when predicting the future position of the objects <NUM>. For instance, it can be predicted that a person would walk around furniture or other obstacles rather than climb over them.

In some examples the apparatus <NUM> can be configured to use information obtained by the LiDAR device <NUM> to predict the locations of areas of interest of the object <NUM> at the given point in time. For example, information from the LiDAR device <NUM> can be used to plot a trajectory of an object <NUM> and from that trajectory predict the position of the area of interest at a future point in time.

In some examples the apparatus <NUM> can be configured to use information obtained by one or more sensors configured to measure movement of the LiDAR device <NUM> to predict the locations of areas of interest of the object <NUM> at the given point in time. In some examples the LiDAR device <NUM> itself could be moving. For instance, the user of the device <NUM> could be walking with the device <NUM> or a vibrator or other motor could be configured to cause movement of the device <NUM> or there could be any other suitable motion. In such examples, information from the sensors could be used to predict a trajectory of the LiDAR device <NUM>.

In some examples the LiDAR device <NUM> could be provided within a vehicle. In such examples onboard vehicle data about the position and trajectory of the vehicle could be used to predict the positions of the object <NUM> relative to the LiDAR device <NUM>. Such data could comprise data about the current velocity of the vehicle, data relating to a planned route of the vehicle and/or any other suitable data.

In examples where the object <NUM> is a person the location of the area of interest of the object <NUM> can be predicted using predictions based on human behaviour. In such examples patterns of human behaviour within a given environment can be established. The patterns can be established by monitoring the movements of people over time. For instance, it can be determined that when people walk in to a particular room or other environment they would tend to walk towards a particular location or object within the environment. Or in other examples if a person is performing a known activity such as a sport then this could be used to predict the movement or motion of the person. This information can then be used to predict the movements of the people within the environment.

In some examples the patterns of human behaviour could comprise complex movements such as hand movements. This could enable the examples of the disclosure to be used to recognize hand gestures or other similar inputs.

Once the location of the area of interest has been predicted for a given time then, at block <NUM>, the method comprises adding a delay to transmission of the measurement location projection pattern <NUM> so that one or more measurement locations of the measurement location projection pattern <NUM> are incident on the at least one area of interest of the at least one object <NUM> at the given time. The delay that is added to the measurement location projection pattern <NUM> increases the time interval between the transmission of two successive measurement location projection patterns <NUM>. This delay, in combination with the relative movement of the LiDAR device <NUM> and the object <NUM>, adds a spatial shift to the measurement location projection pattern <NUM> so as to increase the proportion of measurement locations of the measurement location projection pattern <NUM> that are incident on the area of interest.

The delay that is to be added to the measurement location projection pattern <NUM> can be estimated using any suitable means. In some examples the delay that is to be added to the transmission of the measurement location projection pattern <NUM> can be estimated by calculating an expected position of the area of interest based on the object's <NUM> relative velocity to the LiDAR device. Once the expected position is determined then the method can comprise calculating a spatial shift for the measurement location projection pattern <NUM> based on this relative velocity and the current and desired projected measurement locations. Once the spatial shift has been calculated a corresponding time delay to produce the spatial shift can then be calculated.

The delay can be added to the transmission of the measurement location projection pattern <NUM> by any suitable means. In some examples the delay can be added to the transmission of the measurement location projection pattern <NUM> by controlling power to the LiDAR device <NUM>. For instance, the LiDAR device <NUM>, or components of the LiDAR device can be turned on and off as determined by the delay that has been calculated.

Once the delay has been added to the transmission of the measurement location projection pattern <NUM> this causes the measurement location projection pattern <NUM> to be transmitted at a given point in time. This can increase the gap between successive transmissions of the measurement location projection pattern <NUM>. In some examples the apparatus <NUM>, or any other suitable means, can be configured to interpolate between consecutive measurements detected by the LiDAR device to account for the delay added to the transmission of the measurement location projection pattern <NUM>. The interpolation can be used to help to match information obtained from the LiDAR device <NUM> with other information such as images or videos obtained by the imaging device <NUM>.

In some examples the delay added to the transmission of the measurement location projection pattern <NUM> can be limited to avoid excessive reduction in the temporal resolution of the data that can be obtained from the LiDAR device <NUM>. If it is determined that the delay that is needed exceeds the threshold or limit then the adjustment in the timing of the transmission of the measurement location projection pattern <NUM> can be paused until it is possible to use a smaller delay that is within the threshold.

In some examples both the LiDAR device <NUM> and the object <NUM> could be stationary. In such examples the apparatus <NUM> could be configured to cause movement of the LiDAR device <NUM> so as to provide relative movement between the LiDAR device <NUM> and the object <NUM>. For instance, a vibrator, or motor or other means for providing motion could be provided within the device <NUM>. This could be controlled by the apparatus <NUM> to provide any movement as needed. For instance, the apparatus <NUM> could control the vibrations of the device <NUM> and so generate relative motion between the LiDAR device <NUM> and the object <NUM>.

<FIG> show an example device <NUM> in use. In this example the device <NUM> is a phone or other hand-held user device. Other types of devices could be used in other examples of the disclosure. The LiDAR device <NUM> can be provided within the device <NUM> and so is not shown in <FIG>.

In <FIG> the object <NUM> that is to be measured by the LiDAR device <NUM> is a person <NUM>. The person <NUM> is positioned in front of the device <NUM> so that, at least part of, the measurement location projection pattern <NUM> transmitted by the LiDAR device <NUM> is incident on the person <NUM>. In this example, the area of interest of the person <NUM> could be the user's face. This is the part of the person <NUM> that can be used to enable the person to be identified or to provide any other suitable information.

In the example of <FIG> the measurement location projection pattern <NUM> is a dot projection pattern. The measurement locations <NUM> within the measurement location projection pattern <NUM> comprise a plurality of dots. The plurality of dots <NUM> are evenly spaced in a plurality of rows and columns. Other shapes and/or arrangements for the measurement locations <NUM> could be used in other examples of the disclosure.

In the example shown in <FIG> four rows and columns of the measurement location projection pattern <NUM> are shown. It is to be appreciated that the measurement location projection pattern <NUM> could comprise more measurement locations than this. It is also to be appreciated that the measurement location projection pattern <NUM> and the measurement locations <NUM> are not shown to scale in <FIG>.

In <FIG> the measurement locations <NUM> of the measurement location projection pattern <NUM> are shown. The person <NUM> is positioned relative to the device <NUM> so that the measurement locations <NUM> of the measurement location projection pattern <NUM> fall either side of the person's face. In such circumstances the amount of useful information that could be obtained from the LiDAR measurements could be limited.

<FIG> also shows the ideal measurement locations <NUM> based on the predicted location of the one or more areas of interest. The ideal measurement locations <NUM> are shown more clearly in <FIG> which is an enlarged view of the person <NUM> and the measurement location projection pattern <NUM>. In <FIG> the ideal measurement locations <NUM> are shown as dots with a bold outline while the other measurement locations <NUM> are shown without the outline. As shown in <FIG> the ideal measurement locations <NUM> are aligned with the person's face.

In the example of <FIG> the device <NUM> and the person <NUM> are moving relative to each other as indicated by the arrows <NUM>. This movement can cause a change in position of the person <NUM> relative to the device <NUM>. The movement can be caused by the user of the device <NUM> moving their hand or by any other suitable means.

When the device <NUM> is moving the methods described herein can be applied to predict the location of the one or more areas of interest (in this example, the location of the person's face) relative to the LiDAR device <NUM> at a given point in time and use that information to estimate a delay that is to be added to the transmission of the measurement location projection pattern <NUM> so that at least some of the measurement locations <NUM> within the measurement location projection pattern <NUM> are shifted to the ideal measurement locations <NUM>.

In <FIG> the person <NUM> and the device <NUM> have moved relative to each other compared to the example shown in <FIG>. In <FIG> the person <NUM> is now facing in a different direction. In <FIG> the delay has been added to the measurement location projection pattern <NUM>. The delay can be added by implementing an on/off sequence for the LiDAR device <NUM> or by any other suitable means.

The delay to the transmission of the measurement location projection pattern <NUM> has generated a spatial shift in the position of the measurement locations <NUM> relative to the person <NUM>. The spatial shift is such that some of the measurement locations <NUM> now fall within the area of interest of the person. In this case, at least some of, the measurement locations <NUM> are incident on the person's face. This can enable the LiDAR measurements to obtain information from the person's face. This causes more measurement locations <NUM> to be incident on the one or more areas of interest than otherwise would be without the delay and this can provide for more accurate and reliable information from the LiDAR measurements.

In some examples the effects provided by the examples of the disclosure can increase over time as a plurality of different delays are provided to the transmission of a plurality of measurement location projection patterns <NUM> on a plurality of different occasions. In such examples the relative movement of the object <NUM> and the LiDAR device <NUM> can be monitored over the time period for which the plurality of measurement location projection patterns <NUM> are transmitted. This can enable a larger number of measurement locations <NUM> to be incident on the area of interest and so can increase the resolution of the data obtained by the LiDAR device <NUM>.

In the example of <FIG> the relative movement between the person <NUM> and the LiDAR device <NUM> can be generated by the user moving the device <NUM> and/or by the person <NUM> moving. In other examples there might not be any relative movement between the person <NUM> and the device <NUM>. In such examples, in order to generate the relative movement a vibrator or other motor can be configured to vibrate the device <NUM>. This can cause movement of the device <NUM> and/or the LiDAR device <NUM> as required.

<FIG> shows another example method that could be implemented using the apparatus <NUM> and devices <NUM> described herein.

At block <NUM> the method comprises capturing image data and LiDAR data. The captured image data and LiDAR data can be used to determine information about one or more objects <NUM> within the environment around the device <NUM>. For example, it can be used to determine the location and/or movement of the objects <NUM>. The image data can be obtained using the imaging device <NUM> of the device <NUM> and/or any other suitable means. The LiDAR data can be obtained using the LiDAR device <NUM> of the device <NUM> and/or any other suitable means.

At block <NUM> movement of one or more objects <NUM> within the environment around the device <NUM> is predicted. Any suitable means or combination of means can be used to predict movement of the objects <NUM>. For instance, a trajectory of an object <NUM> could be plotted based on previous motion of the object <NUM>. In examples where the objects <NUM> comprise a person or people then human behaviour patterns could be used to predict the motion of the person or people.

At block <NUM> the movement of the device <NUM> can be predicted. The movement of the device <NUM> can be predicted based on one or more motion sensors such as accelerometers, or any other suitable means. In some examples the apparatus <NUM> can be configured to cause the movement of the device <NUM>, for example the apparatus <NUM> can be configured to activate a motor or vibrator to cause vibration of the device <NUM>. In such examples the movement of the device <NUM> can be predicted from the activation of the motor or the vibrator. In some examples the movement of the LiDAR device <NUM> could be provided by a motorized configurable device <NUM>. For example, the device can be configured to allow for rolling, sliding, folding or other movements of a portion of the device <NUM> comprising the LiDAR device <NUM>. This reconfiguration can be controlled by a motor or any other suitable means.

In some examples the movement of the device <NUM> can be predicted from the context of the device <NUM> and models or patterns of behaviour from the device <NUM>, or other devices, in similar contexts. For instance, if it is determined that the user of the device is performing a gesture that could comprise moving the device <NUM> as they hold it, then this context information could be used to predict motion of the device <NUM>. Similarly if it is determined that the user of the device <NUM> is sitting down then this context would correspond to different expected movements compared to a user who is walking or standing.

The information obtained at block <NUM> and <NUM> can be combined to obtain relative motion of the object <NUM> relative to the device <NUM>. At block <NUM> this information can be used to predict a location of an area of interest of an object <NUM> relative to the LiDAR device at a future point in time. This predicted location of the area of interest provides an indication of the desired location for one or more measurement locations <NUM> at the given point in time.

At block <NUM> the time delay that needs to be applied to the measurement location projection pattern <NUM> is calculated. The time delay is calculated so that when the time delay is added to the measurement location projection pattern <NUM>, at least some of the measurement locations <NUM> are incident on the desired location. The delay can be determined by estimating the spatial shift that needs to be applied to the measurement location projection pattern <NUM> so that, at least some of the measurement locations <NUM> are incident on the desired location.

At block <NUM> the time delay is applied to the transmission of the measurement location projection pattern <NUM>. The time delay can be applied by controlling power to the LiDAR device <NUM> or by any other suitable means. Once the delay is added the measurement locations <NUM> are then incident on the desired location (one or more areas of interest). This can enable more detailed measurements of the object <NUM> or plurality of objects <NUM> to be made.

In the examples shown in the Figures the LiDAR device <NUM> is being used to measure a single object <NUM>. It is to be appreciated that in other examples of the disclosure the LiDAR device <NUM> could be used to measure a plurality of different objects <NUM>. For example, there could be a plurality of people or other objects <NUM> within the environment around the device <NUM>. In such examples the apparatus <NUM> can be configured to detect relative movement between the plurality of objects <NUM> and the LiDAR device <NUM>. This relative movement can be used to predict a location of at least one area of interest for at least some of the plurality of objects <NUM>. In some examples the location of an area of interest could be predicted for all of the objects <NUM>. In some examples the location of an area of interest could be predicted for just a sub-set of the objects <NUM>. Once the location of the areas of interest have been predicted a delay is added to transmission of the measurement location projection pattern <NUM>. The delay is added so as to optimise, or substantially optimise, the incidence of measurement locations <NUM> from the measurement location projection pattern <NUM> on the areas of interest for the at least some of the plurality of objects <NUM>. For instance, it might not be possible for the measurement locations <NUM> to be incident on an area of interest for all of the objects <NUM>, or for all of the objects <NUM> within a subset. In such examples the delay can be added to the transmission of the measurement location projection pattern <NUM> so that as many measurement locations <NUM> as possible are incident on the areas of interest.

In other examples where a plurality of objects <NUM> are to be measured then instead of optimising the delay for the plurality of objects <NUM> as a whole, the delay could be optimised for each object <NUM> sequentially. This could lead to a plurality of different delays being added to the transmission of the measurement location projection pattern <NUM> at different capture instances.

In the examples described herein the object <NUM> has been a person or part of a person. Other types of objects <NUM> could be measured in other examples of the disclosure. For example, the object <NUM> could be vehicles or other devices. The measurement of these objects could be used to enable collisions to be avoided. In some examples the objects could comprise animals or other wildlife. In such examples, patterns of behaviour for the animals or wildlife could be used to predict the future location of the object <NUM> or the area of interest of the object <NUM>. This could help to avoid collisions with the animal or wildlife and/or could enable other useful data to be obtained. In some examples the objects could be inanimate objects that could be moved during their use, for example tools could be used during construction or home improvements that could have one or more moving parts that could be measured using examples of the disclosure.

Examples of the disclosure therefore provide for a method of enabling more accurate or informative measurements of objects <NUM> to be obtained using a LiDAR device <NUM>. As the additional information can be obtained by applying a delay to the transmission of the measurement location projection pattern <NUM> there are no changes that need to be made to the hardware of the LiDAR device <NUM>. Examples of the disclosure could therefore be implemented using any suitable LiDAR devices <NUM>.

Claim 1:
An apparatus (<NUM>) comprising means for:
detecting (<NUM>) relative movement between at least one object (<NUM>) and at least one LiDAR device (<NUM>) controlled by the apparatus (<NUM>), wherein the at least one LiDAR device (<NUM>) is configured to provide a measurement location projection pattern (<NUM>);
predicting (<NUM>) a location of at least one area of interest of the at least one object (<NUM>) relative to the at least one LiDAR device (<NUM>) at a given time; and
adding (<NUM>) a delay to transmission of the measurement location projection pattern (<NUM>) so that one or more measurement locations of the measurement location projection pattern (<NUM>) are incident on the at least one area of interest of the at least one object (<NUM>) at the given time.