Patent ID: 12190454

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

FIG.1schematically illustrates a system for providing visual assistance, hereinafter the system2, to an individual suffering from motion sickness.

Travel sickness, more commonly known as “motion sickness”, can manifest in an individual, during a trip in a vehicle, in the form of various symptoms such as a feeling of discomfort, nausea, dizziness, fatigue, or headaches.

Motion sickness is caused by a contradiction between the visual information and the information provided by the inner ear during the trip. In particular, when the visual attention of the individual is focused on an element that is unmoving within the frame of reference of the vehicle, and therefore unmoving for the individual, the eyes do not perceive movement while movement is perceived by the inner ear, and more precisely by the vestibule.

The system2is suitable for providing the individual with visual information consistent with the movement information collected by the inner ear. More specifically, the system2is suitable for generating an artificial horizon image on the basis of movement data of the vehicle in which the individual is located, then displaying the artificial horizon image in augmented reality to the individual in question. Thus, even when the visual attention of the individual is focused on an element that is unmoving within the frame of reference of the vehicle, the artificial horizon image displayed in real time gives visual movement information that is consistent with what the inner ear perceives.

The system2comprises a sensor3, an artificial horizon device5, and a wearable augmented reality device7. Advantageously, the system2also comprises a database11.

InFIG.1, the artificial horizon device5and the wearable augmented reality device7are two distinct entities. It is understood here, however, that the artificial horizon device5and the wearable augmented reality device7may be one and the same device.

The sensor3is configured for measuring vehicle movement data in real time within a three-dimensional terrestrial frame of reference.

For example, the movement data measured by the sensor3relate to an acceleration of the vehicle in one or more directions, or axes, of the three-dimensional terrestrial frame of reference. The movement data may also relate to an angular velocity of the vehicle in one or more directions, or axes, of the three-dimensional terrestrial frame of reference.

The sensor3may also be configured for collecting other data, such as a geolocation of the vehicle corresponding to a current position of the vehicle. This position may be determined automatically by a server connected to a network to which the sensor3is connected.

Of course, the sensor3may also comprise a geolocation module, not shown here, for calculating geolocation data for the vehicle.

As illustrated inFIG.1, the sensor3comprises at least one accelerometer, here three accelerometers13A,13B, and13C, at least one gyroscope, here three gyroscopes17A,17B, and17C, a memory19, and a processor23.

The accelerometer is configured for measuring data relating to an acceleration of the vehicle within the three-dimensional terrestrial frame of reference.

Because the vehicle can move within a three-dimensional space, it is advantageous to measure the acceleration of the vehicle in each of the three directions of the three-dimensional terrestrial frame of reference. Thus, accelerometer13A measures data relating to the acceleration of the vehicle in a first direction, or along a first axis, accelerometer13B measures data relating to the acceleration of the vehicle in a second direction, or along a second axis, and accelerometer13C measures data relating to the acceleration of the vehicle in a third direction, or along a third axis.

Here, three accelerometers13A,13B and13C have been illustrated. However, it is understood that a single accelerometer can measure the acceleration of the vehicle in each of the three directions.

The gyroscope is configured for measuring data relating to an angular velocity of the vehicle within the three-dimensional terrestrial frame of reference.

As with acceleration, it is advantageous to measure the angular velocity of the vehicle in each of the three directions of the three-dimensional terrestrial frame of reference. Thus, gyroscope17A measures data relating to the angular velocity of the vehicle in the first direction, or around the first axis, gyroscope17B measures data relating to the angular velocity of the vehicle in the second direction, or around the second axis, and gyroscope17C measures data relating to the angular velocity of the vehicle in the third direction, or around the third axis.

Here, three gyroscopes17A,17B, and17C have been illustrated. However, it is understood that a single gyroscope can measure the angular velocity of the vehicle in each of the three directions.

The memory19is configured for storing instructions of a computer program whose execution by the processor23results in the operation of the sensor3.

The memory19may also store the measurements made by the sensor3in real time.

The processor23may be configured for determining a rotation of the vehicle, on the basis of the vehicle movement data.

More precisely, the processor23can determine a vehicle attitude. For an aircraft, the longitudinal attitude in particular is determined, which designates the position of the longitudinal axis of the aircraft relative to the horizontal plane. For a boat, it is rather the horizontal attitude that is determined, also called list, to designate the position of the lateral axis of the boat relative to the horizontal plane.

The artificial horizon device5is suitable for generating an artificial horizon image in real time on the basis of the movement data of the vehicle.

It is understood that the artificial horizon device5is suitable for receiving the movement data measured in real time by the sensor3, and for generating an artificial horizon image on the basis of these movement data.

The artificial horizon image is intended to be displayed to an individual in order to give the individual real-time visual information about the movement of the vehicle via the movement of the artificial horizon displayed. Artificial horizon images are generated in real time to provide visual information to the individual about movements of the vehicle relative to the actual horizon.

As explained above, geolocation data of the vehicle may be determined by the sensor3. In such an embodiment, the artificial horizon device5may be further configured so that it itself determines movement data of the vehicle on the basis of its geolocation. Indeed, the geolocation data can be used to determine a route followed by the vehicle and in this case to anticipate the movement of the vehicle, in particular directional changes.

Of course, the sensor3itself may also determine the movement data of the vehicle on the basis of geolocation data, according to the same principle of determining the route followed by the vehicle and anticipating upcoming movements.

Such an embodiment is applicable in particular to the case where the vehicle in question is an automobile whose movements are constrained in principle by the curves of the road followed by the automobile.

As shown inFIG.1, the artificial horizon device5comprises a configuration interface29, a memory31, and a processor37.

The configuration interface29is suitable for configuring the artificial horizon device5.

Indeed, in addition to the movement data of the vehicle, other information and data may be used to generate the artificial horizon image.

For example, the artificial horizon device5is suitable for generating the artificial horizon image on the basis of a category of the vehicle as well. The artificial horizon device5can thus be configured for one or more of the following categories of vehicles: automobile, train, aircraft, and boat. Indeed, the nature of the movements can be very different from one category of vehicle to another.

Thus, while an aircraft and a boat may be subject to rotations of large amplitude about the lateral axis (the term pitch is then used) and about the longitudinal axis (the term roll is then used), an automobile and a train will be subject to rotations of lower amplitude about these axes. Conversely, a car is generally subject to movements of large amplitude about the vertical axis, and the term yaw is then used, which is less the case for an aircraft.

The configuration interface29thus makes it possible to specify the category of vehicle so that the artificial horizon device5can weight the measured movement data accordingly.

Advantageously, the artificial horizon device5is suitable for generating the artificial horizon image on the basis of data relating to the individual as well. These data include, for example, one or more of the following items of information: age, morphology, visual abilities, and degree of susceptibility to motion sickness.

The effect of motion sickness is indeed variable from one individual to another due to the physical characteristics specific to each person. Thus, for an individual with a high degree of susceptibility, it is necessary for the artificial horizon image to represent the artificial horizon more markedly and to occupy a larger area of the visual field of the individual than for an individual whose degree of susceptibility to motion sickness is low.

Of course, the configuration interface29may include other functionalities and allow the artificial horizon device5to be turned on or off.

The memory31is configured for storing instructions of a computer program whose execution by the processor37results in the operation of the artificial horizon device5.

In particular, the processor37may be configured, similarly to the processor23of the sensor3, for detecting or determining a rotation of the vehicle on the basis of the movement data of the vehicle.

The wearable augmented reality device, hereinafter the wearable device7, is suitable for displaying the artificial horizon image in real time to an individual who is wearing the wearable device and is a passenger of the vehicle. The wearable device7operates on the principle of augmented reality technology, namely the integration or superposition of virtual elements within a real environment. The artificial horizon image is thus combined with reality in the field of vision of the individual wearing the wearable device7.

In other words, the wearable device7is suitable for embedding visual information about movement, manifesting in the form of an artificial horizon in the real image viewed by the individual. Thus, even when all elements present in the visual field of the individual are unmoving within the frame of reference of the vehicle and therefore unmoving relative to the individual, the artificial horizon image informs the individual's eyes of the movement of the vehicle. The obtained result is a better correlation between the information transmitted to the brain by the inner ear and the visual information collected by the individual's eyes.

The augmented reality device7comprises augmented reality glasses or an augmented reality helmet41, possibly a motion sensor43, a memory47, and a processor53.

The augmented reality glasses and the augmented reality helmet are configured for displaying the artificial horizon image, generated by the artificial horizon device5, to the individual wearing the glasses or helmet in question. This type of device is widely known to those skilled in the art and allows superimposing a virtual image and a real image within the wearer's field of vision.

The motion sensor43is configured for measuring movement data relating to the head of the individual wearing the wearable device7.

For example, the motion sensor43is suitable for measuring data on the movement of the individual's head within a three-dimensional frame of reference of the vehicle. Indeed, part of the motion sickness which the individual suffers from may also be due to movements of his/her head. Thus, in addition to the movement data for the vehicle, the system2may also provide, in one or more embodiments, for measuring the movement of the head of the individual wearing the wearable device7.

In general, the operation of the motion sensor43is the same as that of sensor3. Thus, the motion sensor43is configured for measuring an acceleration and an angular velocity of the individual's head within the three-dimensional frame of reference of the vehicle. To carry out such measurements, the motion sensor43may integrate one or more accelerometers and one or more gyroscopes for respectively measuring the acceleration and velocity in one or more directions of the three-dimensional frame of reference of the vehicle.

As explained above, the system2described herein is particularly suitable for an individual who is a passenger of a vehicle, whose visual attention is focused on elements or objects that are unmoving within the frame of reference of the vehicle. Indeed, in such a case, the visual information transmitted by the eyes of the individual is not movement information and is therefore in contradiction to the information transmitted to the brain by the inner ear.

Typically, the individual's attention is focused on the screen of a user terminal, such as a smart phone or digital tablet. Thus, in one or more embodiments, the system2takes into account the presence of a screen within the visual field of the individual, to generate the artificial horizon image.

This particular case is illustrated inFIG.2. The vehicle59considered here is an automobile. The individual is traveling in the automobile59and may suffer from motion sickness during the trip. This motion sickness is all the more probable when, as illustrated inFIG.2, the individual's visual attention may be focused on a screen61. The individual shown in this figure is wearing the wearable augmented reality device7.

Also, in such a case, the wearable device7may further be suitable for measuring data on the position of the individual's head relative to the screen61. The wearable device7is then also suitable for sending these data on the position of the individual's head to the artificial horizon device5so that said device can generate the artificial horizon image also on the basis of these data on the position of the individual's head relative to the screen61and, possibly, also on the basis of data relating to the screen61.

The data relating to the screen61comprise for example data relating to the dimensions of the screen61. This data relating to the screen61may also comprise data relating to a multimedia content displayed by the screen61. These data may for example be supplied to the artificial horizon device5via the configuration interface29.

Advantageously, the wearable device7is also suitable for detecting the presence of the screen61within the individual's field of vision. The wearable device7comprises an image capture and processing module which allows capturing an image and then detecting the presence of a screen in this image. Thus, when the screen61is detected within the visual field of the individual, information about the presence of the screen61is acquired by the wearable device7then sent to the artificial horizon device5. The latter is then configured for taking into account the presence of the screen within the visual field of the individual, to generate the artificial horizon image. Details concerning the use of the screen61, or more precisely of the image of the screen61, will be given later on in the description.

The memory47is configured for storing instructions of a computer program whose execution by the processor53results in the operation of the wearable device7.

Finally, the database11is configured for storing one or more vehicle routes. Within the database11, each route is associated with one or more artificial horizon images.

This embodiment takes advantage of the fact that a vehicle may repeat the same routes, so it is advantageous to keep in the database11routes traveled and artificial horizon images generated by the artificial horizon device5during these routes.

Thus, during a route followed by the vehicle, the route followed is recorded and stored in the database11. This route may in particular be characterized by the movement data measured along the route as well as by the geolocation data measured by the sensor3.

At the same time, the artificial horizon images generated in real time along the route by the artificial horizon device5may be stored in the database11. The artificial horizon images may also be stored in the memory31then stored in the database11only at the end of the route.

In the embodiment in which the system2comprises the database11described above, it is particularly advantageous for the artificial horizon device5to be suitable for detecting, during a current route, whether this route has already been taken and identified in the database11.

The artificial horizon device5is then further suitable for determining a current route of the vehicle, on the basis of the movement data and the geolocation data of the vehicle, and comparing this current route with the route(s) stored in the database11.

The artificial horizon device5is further suitable for:if the current route is already stored in the database11, generating the artificial horizon image on the basis of the artificial horizon image(s) associated with the current route within the database11,otherwise, storing in the database11the current route and the artificial horizon image(s) generated on the current route.

The advantage of such an embodiment is to be able to anticipate the movements of the vehicle and display the artificial horizon image to the individual sooner, which further limits the effects of motion sickness.

A method of providing visual assistance to an individual suffering from motion sickness will now be described with reference toFIG.3.

In the context of implementing the method, an individual who may be subject to motion sickness is traveling in a vehicle. During the trip, the vehicle is moving relative to the actual horizon and this movement information, although collected by the inner ear and then transmitted to the brain, may not be detected by the individual's eyes which in such case are sending incorrect information to the brain. The contradiction between the visual information and the information provided by the inner ear is then the cause of the ailments described above and which constitute motion sickness.

In particular, this absence of visual information is often linked to the fact that nothing in the individual's visual field is indicating movement, which can occur in particular when the individual's visual attention is focused on elements or objects that are unmoving within the vehicle's frame of reference. For example, in the case illustrated inFIG.2, the individual traveling in the automobile59may be focused on the screen61because he/she is viewing multimedia content such as a video. The system2described above then makes it possible to reduce the risks of motion sickness.

In a step S1, the artificial horizon device5of the system2is configured. This configuration is for example carried out via the configuration interface29. This configuration step makes it possible, for example, to indicate the category of the vehicle to the artificial horizon device5. The vehicle may thus be, without limitation, an automobile, a train, an aircraft, or a boat. The vehicle may also be a submarine.

In the example illustrated inFIG.2, the vehicle59in which the individual is located is an automobile.

The information relating to the category of vehicle is particularly useful since the movements of the various vehicles are of different types. As explained above, an aircraft and a boat are mainly subject to rotations about the lateral axis, i.e. pitch, and about the longitudinal axis, i.e. roll. In contrast, an automobile is more likely to execute rotations about its vertical axis.

The specification of the category of vehicle during the configuration step thus allows, for example, appropriately weighting the various movements of the vehicle so that the artificial horizon device5gives an appropriate and weighted significance to each movement of the vehicle in a given direction.

The configuration of the artificial horizon device5may further make it possible to provide data relating to the individual comprising one or more of the following items of information: age, morphology, visual abilities, and degree of susceptibility to motion sickness.

During a step S2, the sensor3measures vehicle movement data. As explained above, the vehicle movement data comprise for example an acceleration of the vehicle within the three-dimensional terrestrial frame of reference and an angular velocity of the vehicle within the three-dimensional terrestrial frame of reference.

In the example illustrated inFIG.2, a three-dimensional terrestrial frame of reference is represented. This frame of reference has a point O as its origin and comprises three orthogonal axes x, y, and z. Typically, if the sensor3is the one illustrated schematically inFIG.1and comprising three accelerometers13A,13B,13C and three gyroscopes17A,17B,17C, then:accelerometer13A and gyroscope17A respectively measure the acceleration and angular velocity in direction x;accelerometer13B and gyroscope17B respectively measure the acceleration and angular velocity in direction y; andaccelerometer13C and gyroscope17C respectively measure the acceleration and angular velocity in direction z.

Furthermore, other measurements may be carried out during this step S2. For example, the sensor3may measure geolocation data of the vehicle. This data may be collected in real time. As explained above, the geolocation data may be determined by the sensor3via a server connected to a network or via a geolocation module integrated into the sensor3.

Still during this step S2, the wearable device7may measure data on the movement of the individual's head within a three-dimensional frame of reference of the vehicle. Such a frame of reference is not illustrated inFIG.2but traditionally allows defining a vertical axis, a longitudinal axis, and a lateral axis. These axes are specific to the geometry of the vehicle and make it possible to characterize in particular the rotational movements of the vehicle, in particular yaw which corresponds to a rotation about the vertical axis, roll which corresponds to a rotation about the longitudinal axis, and pitch which corresponds to a rotation about the lateral axis. These same concepts can also be applied to the movements of the individual's head within the vehicle.

Among the other measurements made, the wearable device7may also measure the position of the individual's head in relation to a screen. Such measurements may be made in particular in the context ofFIG.2in which the individual, who is a passenger of the vehicle59, is likely to be looking at the screen61during the trip, for example in order to view multimedia content.

As explained above, certain embodiments of the invention also allow predictive generation of the artificial horizon image. A first embodiment is described below and corresponds to steps S3, S4, S5, and S6of the method illustrated inFIG.3. A second embodiment is also described below and corresponds to steps S7and S8of the method illustrated inFIG.3. These embodiments may of course be implemented combined.

Concerning the first embodiment: during a step S3, the movement and geolocation data of the vehicle are correlated with the route(s) stored in the database11. This correlation aims to determine whether the current route followed by the vehicle has already been listed in the database11. Each route stored in the database11is referenced by a set of movement data and geolocation data of the vehicle, which allows the system2to compare the current route with stored routes, during step S3.

Thus, during a step S4, the system2determines whether or not the current route is stored in the database11, on the basis of a comparison made between the current route, characterized by the movement and geolocation data of the vehicle, and the route(s) stored in the database11.

During a step S5, implemented if the current route of the vehicle is already stored in the database11, the artificial horizon image(s) within the database11associated with the corresponding route are fetched and sent to the artificial horizon device5.

Otherwise, during a step S6implemented if no route stored in the database11corresponds to the current route of the vehicle, the artificial horizon device5stores the current route in the database11. The current route is stored in the database11with the vehicle movement and geolocation data obtained during step S2. Furthermore, the artificial horizon images generated by the artificial horizon device5during this current route will also be stored in the database11, to be associated with the current route.

Concerning the second embodiment: during a step S7, the geolocation data of the vehicle are used to anticipate, for example using a map accessible to the system2via a network, the route followed by the vehicle.

During a step S8, the artificial horizon device5uses the upcoming route determined in the previous step to predict the future movements of the vehicle.

In the example illustrated inFIG.2, the vehicle concerned is an automobile59. Typically, the movement of the automobile59is constrained by the layout of the roads so that the movement of the automobile59follows the layout of the road on which it is located. Thus, the geolocation data of the automobile59make it possible to determine, via a map, the position of the automobile59and therefore the route followed by the automobile59. It is thus possible, by using the layout of the route followed by the automobile59, to predict the movements of the vehicle, in particular directional changes.

During a step S9, the artificial horizon device5generates an artificial horizon image in real time on the basis of the vehicle movement data. The artificial horizon image allows the individual to be visually informed of the movements of the vehicle in relation to the actual horizon.

For example, the artificial horizon image comprises an artificial horizon line.

Furthermore, other data may be taken into account for generating the artificial horizon image. As explained above, the artificial horizon device5may use information entered via the configuration interface29, such as the category of the vehicle or data relating to the individual.

With reference to the two embodiments described above, the generation of the artificial horizon image may also be predictive, based on the artificial horizon images already generated for the same route and stored in the database11(first embodiment) or by using the upcoming layout of the route followed in order to anticipate the movements of the vehicle relative to the actual horizon and generate the artificial horizon image accordingly (second embodiment).

Moreover, in the case where the visual attention of the individual is focused on a screen, as in the example illustrated inFIG.2, the actual screen image may also be used in order to generate a suitable artificial horizon image.

To this end, as explained above, the wearable device7may comprise an image capture and processing module so that, during implementation of the method, the wearable device7captures a real image in order to detect the presence or absence of a screen within this real image corresponding to the field of vision of the individual wearing the wearable device7.

Also during the method, the wearable device7may determine data relating to the screen, namely its dimensions or the multimedia content displayed. These data relating to the screen are sent to the artificial horizon device5and are used, during step S9, to generate the artificial horizon image.

Thus, when the screen is detected within the individual's field of vision, the artificial horizon device5generates an artificial horizon image comprising a pattern representative of the artificial horizon and that can be superimposed on a portion of an edge of the screen.

Alternatively or in parallel, when the screen is detected within the individual's field of vision, the artificial horizon device generates an artificial horizon image comprising a pattern that can be superimposed on a portion of the screen in the form of tiling distorted according to the artificial horizon.

Finally, during a step S10, the augmented reality device7displays the artificial horizon image in real time to the individual who is wearing the wearable device7and is a passenger of the vehicle.

More precisely, the augmented reality glasses or the augmented reality helmet integrated into the wearable device7display(s) the artificial horizon image to the individual in a manner that superimposes the artificial horizon image on the real image, thus integrating the artificial horizon in real time within the field of vision of the individual who can thus visually perceive movements relative to the actual horizon.

Combinations of artificial horizon images and real images, each forming an image IM viewed by the individual, are illustrated according to different embodiments inFIG.4A,FIG.4B, andFIG.4C.

The screen61appears in each of these images, since each time concerns the case where the screen61is within the individual's field of vision, its presence being detected by the image capture and processing module of the wearable device7.

In the example illustrated inFIG.4A, the artificial horizon image takes the form of an artificial horizon line HA. This line physically indicates the position of the actual horizon, and its position in the artificial horizon image therefore follows the movements of the vehicle. It should be noted that, in this example, the artificial horizon device5does not make use of the presence of the screen within the individual's field of vision in order to generate the artificial horizon image.

In the example ofFIG.4B, this time the artificial horizon image makes use of the presence of the screen61within the individual's field of vision. Thus, the artificial horizon image comprises a pattern MOT that can be superimposed on a portion of an edge of the screen61. The wearable device7superimposes this pattern, during step S10, on the corresponding portion of the edge of the screen61. The advantage of such a pattern MOT is that it does not interfere with the viewing, by the individual, of the content displayed by the screen61. In this example, the pattern MOT makes it possible to view an artificial horizon line which connects the extremities of the pattern MOT, respectively located on the side edges of the screen61.

Finally, in the example ofFIG.4C, the artificial horizon image also uses the presence of the screen61within the individual's field of vision. The artificial horizon image comprises a pattern that can be superimposed on a portion of the screen61in the form of distorted tiling PAV. The distortion varies in real time to characterize the movements of the vehicle relative to the actual horizon. The tiling PAV shown in this figure corresponds to a grid and is superimposed on the screen61.

The present invention offers several advantages.

First of all, the use of augmented reality makes it possible to visually inform an individual, who is a passenger of a vehicle, of the movements of the vehicle in question in relation to the actual horizon even when the visual attention of the individual is focused on elements or objects that are stationary within the vehicle's frame of reference.

Next, the use of geolocation to anticipate the route followed by the vehicle and therefore the upcoming movements allows rapid predictive generation of the artificial horizon image, which limits the lag between the artificial horizon as displayed to the individual via the artificial horizon image and the relative actual position of the vehicle, and therefore of the individual, with respect to the actual horizon.

Finally, the use of the presence of the screen within the individual's field of vision to superimpose the artificial horizon image on the real image, makes it possible to bring the movement information as close as possible to the visual attention of the individual, which is focused on the content displayed on the screen.

Although the present disclosure has been described with reference to one or more examples, workers skilled in the art will recognize that changes may be made in form and detail without departing from the scope of the disclosure and/or the appended claims.