Patent Description:
Helmet-Mounted Display (HMD) systems used in aircraft, for example, are arranged to display image artefacts such as symbols overlain over a user's view of the outside world. Such symbols may include markers indicating the position of features on the ground, a horizon line for example, or they may depict cockpit instruments and flight information in a virtual head-up display intended to appear locked in position relative to the cockpit interior.

In one known example, a tracking system is provided to track the orientation of a helmet at all times. The tracked orientation is supplied to a helmet-mounted display system with sufficient frequency to enable the display system to update the position of symbols so as to appear fixed in space irrespective of movement of the user's head.

If a wearer of the helmet is exposed to vibration or buffeting, for example through a seat, resultant sudden movement of the helmet may cause unacceptable instability in the display of symbols that are intended in such display systems to appear fixed in space.

<CIT> discloses a head-mounted display for displaying an image that matches a viewer's head movement. There is a head tracker for detecting the viewer's head movement, an eye tracker for detecting the viewer's eye movement, and an adaptive filter for adaptively filtering the output of the head tracker in accordance with the output of the head tracker and the output of the eye tracker.

<CIT> discloses a method and apparatus for determining the orientation of an object relative to a moving or moveable platform. The object may for example be a helmet worn by a pilot of an aircraft in which orientation of the helmet relative the aircraft while in flight may usefully be known, in particular when determining the position of space-stabilised symbols being displayed in an associated helmet-mounted digital display system.

<CIT> discloses a method and system for providing an optical see-through Augmented Reality modified-scale display. There is included a sensor suite which includes a compass, an inertial measuring unit, and a video camera for precise measurement of a user's current orientation and angular rotation rate. A sensor fusion module may be included to produce a unified estimate of the user's angular rotation rate and current orientation to be provided to an orientation and rate estimate module.

Example embodiments of the invention will now be described in more detail with reference to the accompanying drawings, of which:.

Helmet-mounted or head mounted display systems, for example a helmet-mounted display system as described in a patent application by the present applicant and published as <CIT>, rely upon the output of an associated helmet/head tracker system in calculating the position of image artefacts in an image for display. A changing orientation of the helmet/head is determined by the tracker system with a frequency sufficient to enable the display system to update the position of particular symbols, for example a horizon line, so as to appear fixed in space relative to the user's view of the outside world, irrespective of movement of the user's head. In particular, in the system described in <CIT>, the accurate positioning of such symbols in the display relies upon the tracking system predicting the orientation of the helmet a short time after the tracker sensors have provided the data for calculating a latest measure of helmet orientation. This is to ensure that latency in the display system in re-calculating the position of particular symbols, to take account of a change in helmet orientation, does not cause the re-positioning of symbols to appear to lag behind the actual position of the wearer's head.

The prediction functionality in the tracker system can be disrupted by certain random patterns of rapid helmet movement, such as may be caused by vibration of a vehicle in which the wearer is travelling. Vibratory displacements may be transmitted through the user's body to their head, so adding involuntary rotary head movement to any voluntary rotary movement of the wearer's head. The combination of voluntary and involuntary rotary movement is detected by the helmet tracker system sensors and the sensed orientation changes are used by the prediction functionality of the tracker system with undesirable results. A new technique is required to enable the effects of vibration to be mitigated from sensor data to be used in the tracker system to predict helmet orientation intended for use by the associated display system.

It may be thought that one possible solution to such a problem is to reduce the latency in the display system so that every movement, whether involuntary or voluntary, may be taken into account when repositioning symbols in the display. However, the latency of the display system is strongly limited by the technology, including speed of processing and response time of display devices, and is difficult to reduce the levels required to respond accurately to vibratory or buffeting movement.

<FIG> illustrates a vibrational system <NUM>. Vibrational system <NUM> comprises a translation vibration <NUM> which acts on a body, such as a user on a seat. The body has a body transfer function <NUM>. The body transfer function <NUM> is the function that when the translation vibration is used as the input, the body transfer function outputs a head angular vibration <NUM>. The body transfer function <NUM> may depend upon the user, and the properties of the seat. Therefore, a solution to mitigate against the vibrational translation of the user is to attempt to estimate the body transfer function.

<FIG> illustrates a head tracker system <NUM> in accordance with some examples. Head tracker system comprises at least one vibration sensor <NUM>, a non-vibrational head tracker <NUM>, and an adaptive filter <NUM>.

A vibration of a platform causes vibration of the user which may be determined by the at least one vibration sensor <NUM>. The vibration sensor <NUM> outputs a force variable. The at least one vibration sensor <NUM> may be located at or close to a point of contact between a user and the platform such that the vibrational forces imparted onto the body may be determined based on the force variable output of the at least one vibration sensor <NUM>.

As described with relation to <FIG>, by estimating the body transfer function <NUM> the intended orientation of the users head can be calculated. However, as stated above, the non-vibrational head tracker <NUM> is not suited to mitigate against the vibrational translation of the platform. The non-vibrational head tracker <NUM> outputs a non-predicted orientation of the user's head at a current time, i.e. an output of the orientation at the current time, rather than at the orientation at a time ahead of the current time.

The force variable is used as an input to the adaptive filter <NUM>. The filter coefficients are based on a comparison <NUM> of the output of the adaptive filter <NUM> and output from the non-vibrational head tracker <NUM>. The output of adaptive filter <NUM> provides an estimate of the body transfer function. The estimated body transfer function may be used to derive the intended orientation of the user.

In some examples the vibration sensor <NUM> may comprise an accelerometer. In some examples the vibration sensor <NUM> may comprise a velocity sensor, and the output may be differentiated to obtain an acceleration. In some examples the vibration sensor <NUM> may comprise a displacement sensor, and the output of the sensor may be double differentiated to obtain an acceleration. In some examples where the output of the sensor is differentiated, the differentiated output may also be filtered using a band pass filter to remove noise.

In some examples the banbass filter on the accelerometer may limit the frequency to <NUM>. <NUM> this is above the frequency in which significant vibration is transferred to the head and is also below half the update rate of typical systems (typically greater than <NUM>). The lower frequency is set to a value which will depend on the exact low frequency dynamics required and typically is <NUM>.

In some examples the adaptive filter <NUM> may receive a measure of the force derived from the output of the vibration sensor <NUM>. In some examples the filter may receive a measure of acceleration derived from the output of the vibration sensor <NUM>.

In some examples the non-vibrational head tracker <NUM> may comprise a gyroscopic sensor. The gyroscopic sensor may output a predicted or non-predicted orientation of the head of the user, and/or a predicted or non-predicted rate of change of orientation or pose of the user. In some examples the non-vibrational head tracker <NUM> may comprise a system similar to that described in <CIT>.

In some examples the adaptive filter <NUM> may comprise a recursive least squares (RLS) filter, where the constants of the RLS filter as based on the output of the comparison <NUM>.

In some examples the comparator <NUM> may calculate a difference between the output of the adaptive filter <NUM> and the output of the non-vibrational head tracker <NUM>.

<FIG> illustrates a head vibration prediction system <NUM> in accordance with some examples. The head vibration prediction system <NUM> is similar to the head tracker system <NUM> described with reference to <FIG>. Similar features use the same reference signs as in <FIG>.

Head vibration prediction system <NUM> comprises a vibration sensor <NUM>, a delay <NUM>, an adaptive filter <NUM>, a FIR filter <NUM>, a predictive non-vibrational head tracker <NUM>, and a comparator <NUM>.

Vibration sensor <NUM> is configured to measure vibration at a contact point of the platform to a user of the head mounted display and determine a force variable associated with the vibration. The predictive non-vibrational head tracker <NUM> determines a predicted orientation of the user at a predetermined prediction time ahead of the current time, and may be similar to head trackers as described in <CIT>.

A delay <NUM> substantially equal to the predetermined prediction time is added to the output of the vibration sensor <NUM>. The delayed vibration sensor output is provided as an input to the adaptive filter <NUM>, where the coefficients of the filter are based on the output of the comparator <NUM>. The comparator <NUM> receives as inputs the output of the predictive non-vibrational head tracker <NUM> and the adaptive filter <NUM>. The addition of the delay forces the adaptive filter <NUM> to predict the body transfer function at a time ahead of the current time.

The filter coefficients from the adaptive filter <NUM> are provided to the FIR filter <NUM>, and the non-delayed output of the vibration sensor <NUM> is provided as input to the FIR filter <NUM>, which provides as an output the current head vibration.

The effect of the head vibration prediction system <NUM> is that the adaptive filter <NUM> provides a prediction of the body transfer function a predetermined time ahead of the current time, that time equal to the delay <NUM>. The filter coefficients from the adaptive filter <NUM> are then output to the FIR filter <NUM>, which provides the predicted head vibration without substantial lag, as any lag is counteracted due to the fact that the adaptive filter <NUM> predicts the body transfer function.

In some examples a filter may be applied to the output of the vibration sensor <NUM> and/or the output of the predictive non-vibrational head tracker <NUM>. The filter may be a bandpass filter. The filter may be used to remove noise which may be a result of the measurement or determination of the force at the contact point or pose/head orientation.

In some examples the adaptive filter <NUM> may comprise a RLS filter.

In some examples the predictive non-vibrational head tracker <NUM> may comprise at least on gyroscopic sensor. In some examples the predictive non-vibrational head tracker <NUM> may comprise a head tracker as described in <CIT>.

In some examples the comparator <NUM> may calculate a difference between the output of the adaptive filter <NUM> and the output of the predictive non-vibrational head tracker <NUM>.

<FIG> illustrates a combined predicted head orientation tracker system <NUM> in accordance with some examples. The combined predicted head orientation tracker system <NUM> is similar to the head vibration prediction system <NUM> described with reference to <FIG> and similar features use the same reference signs.

Combined predicted head orientation tracker system <NUM> comprises a predictive non-vibrational head tracker <NUM>. Predictive non-vibrational head tracker is configured to output a predicted head orientation <NUM> and a non-predicted head orientation. Predicted head orientation <NUM> is a prediction of the head orientation a predetermined prediction time ahead of the current time. Non-predicted head orientation <NUM> is the head orientation obtained without the prediction.

A delay <NUM>, equal to the predetermined prediction time, is added to the predicted head orientation <NUM>, the delayed predicted head orientation is then compared by a comparator <NUM> with the non-predicted head orientation <NUM>. As the delay <NUM> is equal to the prediction time, the output <NUM> of the comparator <NUM> is equal to the error of the predicted head orientation <NUM>. The comparator <NUM> may find the difference between the delayed predicted head orientation and the non-predicted head orientation <NUM>.

The head orientation may be expressed in terms of Cartesian coordinates, or for convenience with the yaw, roll, and pitch angles of aircraft may be expressed in polar coordinates.

The combined predicted head orientation tracker system <NUM> also comprises a plurality of seat vibration sensors <NUM>. Each seat vibration sensor <NUM> is configured to determine a force variable, related to the force at the contact point between the seat and the user. The contact point may be substantially fixed in positon relative to the user, however it is to be understood that the movement of the contact point may depend on the freedom of movement of the user. Some applications require the user to more securely restrained to a seat than other applications. Furthermore, the contact point may be modelled as a point, however in practical applications the force may act over a finite area.

The seat sensors <NUM> may comprise a force sensor, an acceleration sensor, a velocity sensor or a position sensor. The seat sensor <NUM> or associated device may be configured to convert the received information into a measure of force.

The combined predicted head orientation tracker system <NUM> also comprises a plurality of vibration prediction blocks <NUM>, one vibration prediction block <NUM> for each axis of rotation (e.g. at least one of pitch, roll, and yaw), alternatively there may be one block <NUM> for each orthogonal direction/axis. Each separate vibration prediction block <NUM> receives force variable information from at least one of the plurality of seat vibration sensors <NUM> and orientation information form the predictive non-vibrational head tracker <NUM> as illustrated in <FIG>.

As each vibration prediction block behaves in a similar way, it will only be described for a single vibration prediction block <NUM>. The received force variable in three dimensions is filtered using a band pass filter <NUM>, to remove noise from the sensor information. The filtered force variable is then delayed using delay <NUM>, where the delay <NUM> is equal to the prediction period of the predictive non-vibrational head tracker <NUM>. The delayed force variable is then provided as an input to a set of three RLS filters <NUM>. Each RLS filter receives one of the x, y or z coordinate information (or any other orthogonal coordinates, such as polar coordinates).

The output of the three RLS filters <NUM> is subtracted from the output <NUM> of the comparator by summer <NUM>, and the output of summer <NUM> is used to obtain the filter coefficients of the three RLS filters. The coefficients are then provided to a plurality of FIR filters <NUM>, which have as inputs the filtered seat sensor output, without any delay. The output of the FIR filters <NUM> are then summed by summer <NUM> to provide a vibration correction <NUM>.

The vibration correction <NUM> is then summed with the predicted head orientation <NUM> form the predictive non-vibrational head tracker <NUM> by summer <NUM> to obtain a combined predicted head orientation <NUM>.

The equations below may be used to define the input and output of the filters. However, they are merely one example of equations to define the output of the filters. Other representations of rotations, such as a quaternion representation may be used.

The required correction in prediction orientation is given by: <MAT> where <MAT> is the non-predicted orientation and <MAT> is a delayed version of the predicted output.

For small angle rotations (ωz,ωy,ωx) around the <NUM> orthogonal axes, the corresponding rotation matrix can be approximated by: <MAT>.

Thus by equating equations <NUM> and <NUM>, the 'desired output' (ωz,ωy,ωx) of the <NUM> RLS filters can be obtained.

Each set of the <NUM> input RLS filters may have outputs yi is defined by the following: <MAT> <MAT> <MAT> <MAT> <MAT> <MAT> <MAT>.

For small angles the corresponding rotation matrix ( <MAT>) can be obtained by equation <NUM>, where ωi is the output from the RLS filter (yi).

The current corrected tracker prediction output is given by: <MAT>.

A second solution, in combination with the first, may be to use head angle prediction. A known tracker system, such as that referenced above, uses gyroscopic sensors of helmet movement to provide rate data to a prediction algorithm for use in predicting helmet orientation at some future time. However, to achieve good prediction (without suffering undesirable image artefacts) the prediction time needs to be limited. Given the minimum achievable latency, further improvement is needed to the prediction algorithm in order to improve the stability of symbology. Stability of the symbols is a key factor of the performance and user acceptability criteria; symbols need to look as though they are locked to a fixed position in space irrespective of helmet movement, whether voluntary or involuntary.

It will be understood that components described above with reference to <FIG> may in practice be implemented by a single chip or integrated circuit or plural chips or integrated circuits, optionally provided as a chipset, an application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), digital signal processor (DSP), graphics processing units (GPUs), etc. The chip or chips may comprise circuitry (as well as possibly firmware) for embodying at least one or more of a data processor or processors, a digital signal processor or processors, baseband circuitry and radio frequency circuitry, which are configurable so as to operate in accordance with the exemplary embodiments. In this regard, the exemplary embodiments may be implemented at least in part by computer software stored in (non-transitory) memory and executable by the processor, or by hardware, or by a combination of tangibly stored software and hardware (and tangibly stored firmware). Although at least some aspects of the embodiments described herein with reference to the drawings comprise computer processes performed in processing systems or processors, the invention also extends to computer programs, particularly computer programs on or in a carrier, adapted for putting the invention into practice. The program may be in the form of non-transitory source code, object code, a code intermediate source and object code such as in partially compiled form, or in any other non-transitory form suitable for use in the implementation of processes according to the invention. The carrier may be any entity or device capable of carrying the program. For example, the carrier may comprise a storage medium, such as a solid-state drive (SSD) or other semiconductor-based RAM; a ROM, for example a CD ROM or a semiconductor ROM; a magnetic recording medium; optical memory devices in general; etc..

Claim 1:
A method for determining an intended orientation and/or position of a user's head when the orientation of the user's head is being changed involuntarily by vibratory or buffeting forces transmitted through the user's body from a point of contact of the user's body with said forces, the method comprising:
determining, at a sensor (<NUM>, <NUM>), a sensed force variable at the point of contact due to said forces;
predicting an involuntary component of orientation of the user's head at a predetermined time due to the sensed force variable, the predicting using an adaptive filter (<NUM>) comprising a first input, a first output, a filter coefficient input, and at least one filter coefficient,
characterised in that the first output is equal to the involuntary component of orientation and the first input comprises the sensed force variable;
determining, using at least one tracking system, an orientation variable of the user's head at the predetermined time;
subtracting a first value from a second value (<NUM>), the first value equal to the first output and the second value equal to the orientation variable, to provide a third value equal to the filter coefficient input;
updating the at least one filter coefficient based on the filter coefficient input; and
determining the intended orientation and/or position of a user's head based on the third value;
wherein the point of contact is substantially fixed in location relative to the user.