Patent Description:
Portable eye tracking devices for eye tracking and/or to detect one or more gaze-related parameters of a user, for example in the form of head-wearable spectacle-like systems offer many advantages over stationary, remote eye tracking systems when it comes to the range of possible application scenarios and use cases. Outdoor activities like sports or tasks like operating large machinery are examples for eye tracking application areas in which free head and body movement is required and for which head-worn devices are thus the solution of choice. At the same time, such use cases often also require the wearer to wear specialized eyewear, like protective eyeglasses of various kinds. Pilot visor helmets, skiing- or swimming-goggles, sports helmets, head-band structures and welding protective googles are further examples. Further, physiological features like head or nose shapes of the wearers can vary in wide ranges, e.g. as a function of ethnicity or age.

Existing portable devices traditionally provide integrated spectacle-type systems which do not provide enough flexibility to fit different specialized scenarios and/or are not suitable for children.

Document <CIT> describes an eye tracking module for attachably complementing a head-wearable device of a user allowing for flexibility and/or adaptability to different use cases.

<CIT> discloses a camera module for complementing a frame of a head-wearable device to form a head-wearable eye tracking device, with a base structure having upper portion, first and second legs with first and second eye camera and a scene camera.

However, there is still room to further improve portable eye trackers and head-wearable eye tracking devices, respectively, in particular with regard to manufacturing, long term mechanical stability, reproducibility and/or accuracy.

According to an embodiment of a camera module for complementing a support structure of a head-wearable device to form a head-wearable eye tracking device, the camera module includes a one-piece base structure which includes an at least semi-transparent material, typically a substantially transparent material and includes an upper portion, a first leg portion and a second leg portion. A camera connecting structure is embedded in the one-piece base structure. A first eye camera is connected with the camera connecting structure, and arranged in and/or on the first leg portion. A second eye camera is connected with the camera connecting structure, and arranged in and/or on the second leg portion. A scene camera is connected with the camera connecting structure, and arranged in and/or on the upper portion.

The first eye camera may be used for taking (recording) first images of at least a portion of a first eye of the user wearing the head-wearable eye tracking device, e.g. a left eye, of the user, and the second eye camera may be used for taking (recording) second images of at least a portion of a second eye of the user, e.g. of a right eye of the user.

Within this specification the terms "first camera", "first eye camera" and "left camera" are used synonymously. Likewise, the terms "second camera", "second eye camera" and "right camera" are used synonymously herein.

The first leg portion and the second leg portion, are typically implemented as comparatively thin, elongated elements, in particular respective camera arms. For example, the respective leg portion may have a thickness of less than about <NUM> or even <NUM>, at least close to the upper portion.

The base structure and the legs are designed in this way to avoid any (unwanted) contact of the leg portions with the user's nose or face, and/or to reduce occlusion of the visual field of the user.

The camera module is typically not configured to shape a nose bridge portion, instead the support structure of the complementing head-wearable device may contain a nose bridge portion.

In this regard it is noted that making the first leg portion and the second leg portion sufficiently transparent and thin, and not part of the support structure and the nose bridge portion, respectively, allows for camera placement further towards the forward-facing lines of sight of the user while keeping both product design compact and occlusion low or even minimal.

The recorded eye images do not necessarily need to be a picture as visible by the human eye, but can also be an appropriate representation of the recorded (filmed) eye in a range of light non-visible for humans.

The recorded eye images may e.g. be used for determining one or more gaze-related parameters of a user wearing a head-wearable eye tracking device provided with the camera module. This may even be done by a controller of the camera module, which in these embodiments may form an eye tracking module, but also by an external device, e.g. a companion device such as a tablet or smartphone connectable with the camera module, typically via the head-wearable device.

The gaze-direction related parameter may be a gaze direction, a cyclopean gaze direction, a 3D gaze point, a 2D gaze point, a visual axis orientation, an optical axis orientation, a pupil axis orientation, and a line of sight orientation of the user.

The gaze-related parameter may be a gaze-direction related parameter or a position and/or an eyelid closure, a pupil area, a pupil size, a pupil diameter, a measure of cognitive load, a sclera characteristic, an iris diameter, a characteristic of a blood vessel, a cornea characteristic of at least one eye, a cornea radius, an eyeball radius, a distance pupil-center to cornea-center, a distance cornea-center to eyeball-center, a distance pupil-center to limbus center, a cornea keratometric index of refraction, a cornea index of refraction, a vitreous humor index of refraction, a distance crystalline lens to eyeball-center, to cornea center and/or to corneal apex, a crystalline lens index of refraction, a degree of astigmatism, an orientation angle of a flat and/or a steep axis, a limbus major and/or minor axes orientation, an eye cyclo-torsion, an eye intra-ocular distance, an eye vergence, a statistics over eye adduction and/or eye abduction, and a statistics over eye elevation and/or eye depression, data about blink events, drowsiness and/or awareness of the user, parameters for user iris verification and/or identification.

The scene camera may be used for taking images of a field of view of the user wearing the head-wearable device complemented with the camera module. In the following, the images of the field of view of the user are also referred to as scene images. Accordingly, the scene camera is not to be understood as a camera for taking a picture of an eye of the user.

For reasons of accuracy, the scene camera may have a resolution of at least <NUM> x <NUM> pixels or at least <NUM> x <NUM> pixels, more typically of at least <NUM> x <NUM> pixels, and even more typically of at least <NUM> × <NUM> pixels or at least <NUM> × <NUM> pixels (at least VGA SVGA or HD).

The resolution of the eye cameras and the images may be lower compared to the scene camera, e.g. be at most 400x400, or even at most 200x200.

Corresponding scene and eye images may e.g. be used to determine what the user is looking at. Again, this may be done by the camera module or by an external device.

As the cameras of the camera module are firmly connected with the one-piece base structure, the (relative) camera poses, i.e. the positions and orientations of the cameras with respect to each other may be particularly precisely fixed during manufacturing in a comparatively efficient and/or simple way also ensuring mechanical long-term-stability and reproducibility. This is advantageous for achieving high eye tracking accuracy and high accuracy in determining eye parameters, respectively. For example, the knowledge of the relative eye camera poses, i.e. the relative positions and orientations of the first and second eye cameras are shown to be important for determining eye parameters in document <CIT>.

The transparency of the material of the one-piece base structure typically refers to a transparency in an infrared (IR) - light range, in particular a near-IR light range, and/or a visible light range. In particular, the substantially transparent material of the one-piece base structure is typically substantially transparent in an IR-light range, in particular a near-IR light range, and/or for light in a visible range.

Further, the substantially transparent material of the one-piece base structure is typically a solidified material, in particular a respective (cured) silicone or plastic.

This may facilitate manufacturing because the cameras as well as optional light sources may substantially or even completely be embedded in the material.

Further, using a substantially transparent and typically substantially non-reflective (e.g. a matt surface) material (in the visible light range) for the one-piece base structure facilitates reducing or even avoiding any (noticeable) obstruction of the visual field of a user wearing the head-wearable device complemented with the camera module.

The typically watertight camera module may, in particular, be manufactured by overmolding also allowing for forming particularly thin layers overlaying the camera connecting structure, in particular of parts thereof that are arranged (typically embedded) in the first leg portion and the second leg portion allowing for a particularly lightweight and/or unobtrusive camera module, the camera objectives, and/or the optional light sources for illuminating the user's eyes.

Accordingly, the camera connecting structure (typically including the cameras) is typically molded into (cast in) the one-piece base structure.

Manufacturing the camera module by overmolding also allows to long term stably embed the first eye camera, the second eye camera, and the scene camera at least partly in the one-piece base structure with defined camera poses, in a reproducible, cost-effective manner also ensuring water tightness.

At least one of, typically all of the first eye camera, the second eye camera, and the scene camera may be mechanically and electrically connected to the camera connecting structure.

Further, the camera connecting structure is typically at least partially flexible. This may facilitate manufacturing the camera module in a mold providing fixation structures for the cameras during molding. This is explained in more detail below.

The camera module overmold material may also not be substantially stiff but allow elastic deformations of parts of the camera module during handling. When installed into the support structure, the camera module may return to its original shape.

The camera connecting structure (inside the camera module) may include a circuit board, more typically a respective circuit board for each of the first eye camera, the second eye camera, and the scene camera.

The camera connecting structure (inside the camera module) may in addition include a further circuit board for receiving respective images (image data) from each of the first eye camera, the second eye camera, and the scene camera. The further circuit board may be equipped with electronic components to control image recording, forward, preprocess and/or even process the received image data.

The further circuit board may form or be equipped with a controller providing desired control and/or image processing functionality.

The further circuit board and the controller, respectively, are typically completely embedded within the one-piece base structure, and may be arranged in the upper portion, typically above the scene camera and in an uppermost part of the one-piece base structure, respectively.

The scene camera is typically also arranged in the upper portion of the camera module and the one-piece base structure, respectively.

The controller may be configured for at least one of:.

To provide a desired flexibility during manufacturing, a respective wire connection between the circuit boards may be used.

Alternatively or in addition, the camera connecting structure may be implemented and/or include one or more flexible circuit boards, in particular a flex PCB or a rigid-flex PCB.

The base structure is typically, at least in a lower portion, shaped to partly surround a nose of the user, in particular a nasal bridge of the user's nose, and at least substantially mirror symmetric with respect to a (virtual) central plane of at least the lower portion typically including two leg portions.

In particular, the lower portion typically includes a first (left) leg portion and a second (right) leg portion which are at least substantially mirror-symmetric with respect to the symmetry plane.

The camera module is typically formed such that the first leg portion and the second leg portion are spaced apart from at least one of, typically both of the user and the support structure when the user is wearing the head-wearable eye tracking device.

Further, the first eye camera may be arranged on and/or in the first leg portion, and the second eye camera may be arranged on and/or in the second leg portion.

Typically, at least one of a first angle between an optical axis of the first eye camera and the (vertical) symmetry plane, and a second angle between an optical axis of the second eye camera and the symmetry plane is in a range from about <NUM>° to <NUM>° (or -<NUM>° to about -<NUM>°), more typically about <NUM>° (or -<NUM>°), i.e. <NUM>°+/- <NUM>° (or -<NUM>°+/- <NUM>°). Further, at least one of a third angle between the optical axis of the first eye camera and a (horizontal) plane perpendicular to the (vertical) symmetry plane, and a fourth angle between the optical axis of the second eye camera and this plane is about <NUM>°+/- <NUM>°. Accordingly, an optimal view of the eye cameras onto an eyeball of the user is enabled.

Furthermore, the first angle may at least substantially correspond to the second angle. Likewise, the third angle may at least substantially correspond to the fourth angle.

The one-piece base structure typically also provides an outer surface of the camera module. Thus, the one-piece base structure, which is in the following also referred to as base structure for short, may not only provide supporting functionality but also provides the function of a housing for protecting the included (embedded) components.

Typically, the outer surface of the one-piece base structure is at least substantially mirror symmetric with respect to the symmetry plane.

Further, the scene camera is typically at least substantially centred with respect to the symmetry plane. An optical axis of the scene camera may at least substantially be arranged in the symmetry plane, or at least close to the (central) symmetry plane and/ or close to (expected) eye midpoint of the user. This facilitates a compact design. Furthermore, the influence of parallax error for gaze prediction may be significantly reduced this way.

Likewise, the first eye camera and the second eye camera are typically arranged at least substantially mirror-symmetrically with respect to the symmetry plane.

Placing the eye camera(s) close to the (central) symmetry plane also facilitates a comparatively compact camera module.

The camera module may fit into an enveloping cuboid with dimensions of at most about <NUM> x <NUM> x <NUM> or even at most about <NUM> x <NUM> x <NUM>.

A first distance between the first eye camera and the symmetry plane may be at most about <NUM>. Further, a second distance between the second eye camera and the symmetry plane at least substantially corresponds to the first distance and/or is at most about <NUM>.

An absolute difference between the first distance and the second distance may be at most about <NUM>, more typically at most about <NUM> or even <NUM>.

At least one of the first distance and the second distance may be at most about <NUM>, at most about <NUM>, or even at most about <NUM>.

When the camera module is attached to the support structure of the head-wearable device, the field of view of the wearer may not be occluded, at least not discernible for the wearer. Note that an obstruction of the visual field of the user may essentially not be noticeable by the user since the human mind mostly ignores any obstruction in the nose area of the face, because the nose itself forms a permanent obstruction of the visual field.

The camera module is typically attached to the central portion so that relative camera poses of the first and second eye cameras remain at least substantially fixed even when the central portion of the support structure is deformed under normal wearing conditions.

For this purpose, the camera module may provide a fastenings means typically fitting with a fastener and arranged at least close to the symmetry plane.

The camera module may include one, typically two fastening means for fastening the one-piece base structure at the support structure and/or releasably attaching the one-piece base to the support structure of the head-wearable device, in particular two respective mechanical or magnetic fastening means.

The one or more fastening means may be arranged next to an optical entrance of the scene camera, e.g. an objective lens of the scene camera.

The one or more mechanical fastening means may e.g. be implemented as (through) holes in the base structure (to allow screws as fasteners).

The camera module may be removably attached with screws, snaps or magnets to the supporting structure of the head-wearable device. The connection of the camera module to the support structure may be made such that flexing and bending of the support structure (e.g. a frame) does not or almost not bend or flex the camera module. Mechanical decoupling may be at least facilitated or even fully achieved by one or more fasteners being concentrated in a comparatively small central area of the support structure rather than spread across an area of the support structure that may be subject to bending and flexing. Furthermore, mechanical decoupling may be facilitated or even fully achieved because the camera module arms, in particular in the area of the eye cameras are not (directly) connected to the supporting structure.

Instead of directly connecting to the supporting structure, the camera module may connect to an interposing carrier. This optional interposing carrier may further facilitate mechanical decoupling. Typically, the carrier is attached to the central portion of the support structure permanently. In these embodiments, the camera module may be removably attached with screws, snaps or magnets to the carrier that may be screwed and/or glued to the central portion.

Typically, a distance between the two mechanical fastening means is at most about <NUM> or even <NUM>, or even only <NUM>.

This facilitates reducing any transmission of deformations between the support structure of the head-wearable device and the camera module.

Typically, the first eye camera, the second eye camera, and the scene camera are mechanically and electrically connected to the camera connecting structure.

The camera connecting structure may include a circuit board, more typically a respective circuit board for each of the first eye camera, the second eye camera, and the scene camera.

Typically, a further circuit board typically equipped with a controller and/or configured for receiving respective images from each of the first eye camera, the second eye camera, and the scene camera is provided, in particular in an uppermost part and above the scene camera, respectively.

To facilitate manufacturing, the camera connecting structure may be at least partially flexible.

For this purpose, a wire connection may be provided between the circuit boards.

Alternatively or in addition, the camera connecting structure may be implemented as and/or include a flexible circuit board, in particular a flex PCB or a rigid-flex PCB.

According to an embodiment of a head-wearable eye tracking device, the head-wearable eye tracking device includes a support structure comprising a central portion which is, when the head-wearable eye tracking device is worn by a user, arranged closest to and/or on a nose of the user, and a camera module as explained herein which is fastened to the central portion so that the first leg portion and the second leg portion camera module are at least substantially mechanically decoupled from the support structure. Accordingly, (eye) camera poses are not substantially affected by deformation / bending of the support structure.

This may be achieved by a carrier interposed between the support structure and the camera module, symmetrically attaching the carrier and/or the camera module to (the central portion of) the support structure (in an upper part and/or with respect to the scene camera), and/or by shaping the camera module (and the central portion) so that the first leg portion and the second leg portion are spaced apart from the user (also to avoid deflection of the respective leg portion that may result from contact with the user's skin) and/or the support structure when the user is wearing the head-wearable eye tracking device.

Typically, the central portion is configured to provide (in particular shaped in a lower part as) a nose bridge portion of the user's nose.

In this regard it is noted that, not only for reasons of high wearing comfort, mechanical decoupling of the eye imaging components and the head-wearable device is preferred compared to making the structures of the head-wearable device stiffer.

According to an embodiment, the head-wearable eye tracking device includes a support structure comprising a central portion which is, when the head-wearable eye tracking device is worn by a user, arranged closest to and/or on a nose of the user, and/or provides a nose-bridge portion, and a camera module including a one-piece base structure with an upper portion, a first leg portion, and a second leg portion. A first eye camera is arranged in and/or on the first leg portion, a second eye camera is arranged in and/or on the second leg portion, and a scene camera is arranged in and/or on the upper portion. The camera module is fastened to the central portion in an at least substantially mirror symmetric way with respect to a symmetry plane of at least one of, typically both of the support structure and the camera module (or even the head-wearable eye tracking device), an optical axis of the scene camera being at least substantially parallel to the symmetry plane and/or arranged within the symmetry plane. Typically, two fastening means arranged mirror-symmetric with respect to the symmetry plane are used.

Alternatively or in addition, a carrier is mechanically connected with and arranged between the upper portion of the one-piece base structure and the central portion of the support structure, and is configured to reduce a transmission of a deformation of the central portion of the support structure to the upper portion of the camera module.

Due to the interposed carrier, deforming of the upper portion of the camera module and thus of the whole camera module due to a deformation of the central portion, which may be due to external loads acting on the support structure and/or changes in temperature, is at least reduced. In particular, the impact of any deformation of the support structure, for example a bending of the support structure such as a spectacle frame, on the relative camera poses may also be at least reduced. This allows for also high accuracy under varying conditions, and may in particular be advantageous if calibration-free or hardware-calibrated eye tracking is desired.

Typically, the carrier is mechanically connected with and arranged between the upper portion and the central portion for at least reducing a deformation of the upper portion when the central portion is deformed (under stress), in particular due to external loads acting on the central portion of the support structure such as bending and/or torsional forces which may, for example, be transmitted through spectacle temples.

The carrier is typically mechanically connected with and arranged between the upper portion and the central portion to reduce transmitting at least one of, typically all of a (mechanical) tension, a (mechanical) compression, a bending force and a torque.

The camera module may be any of the camera modules as explained herein and/or a camera module manufactured as explained herein.

The carrier may be releasably connected with the upper portion of the one-piece base structure and the central portion of the support structure.

Further, the carrier may include an at least substantially central trough hole for the scene camera. Alternatively, the carrier may have a sufficiently transparent region for the optical entrance of the scene camera.

The carrier may be fastened to the central portion in a mirror symmetric way with respect to the at least substantially central trough hole and the transparent region, respectively.

Further, the carrier may be implemented as a circuit board, in particular a PCB.

Typically, a plug-socket connection is provided between the camera module and the carrier or the camera module and central portion of the head-wearable device (in embodiments without the carrier). In particular, two respective plug-socket connections, which are arranged mirror symmetric with respect to the symmetry planes of the device, the camera module and/or the carrier, and a common the symmetry plane in the assembled state, may be used for the connection.

The plug-socket connection(s) typically includes (pairs of) one or more spring-loaded pins, in particular pairs two or three spring-loaded pins.

Furthermore, the head-wearable eye-tracking device may include an external electric connector for power supply and/or data exchange (also referred to as second electric connector), and an internal wiring electrically connecting the second electric connector with an internal electric connector (also referred to as first electric connector) and/or the plug-socket connection(s).

The second electric connector may be used for power supply from and/or data exchange with a companion computing device.

The second electric connector may be a USB-connector such as an USB-C-connector. However, the second electric connector may also be a connector for another external computer bus, in particular a computer bus and power connector such as a Thunderbolt-connector (Thunderbolt <NUM> uses USB-C connectors) or a Lightning connector (typically supporting the USB-protocol) or even an Ethernet-connector (Power over Ethernet).

Further, the second electric connector may be arranged at a dorsal end of a frame of the head-wearable device providing the support structure, and/or of a temple of the head-wearable device, respectively. Thereby a connection can be established in an unobtrusive way and wearing comfort may even be increased.

Via the second electric connector (and the wired connection within the head-wearable device), the camera module may be connected with a (mobile) computing companion device such as smartphone, a tablet or the like, supplied with electric power from the companion device and even controlled by the companion device, for example by an app running on the companion device.

The head-wearable device may be a spectacle device, a pilot visor helmet, an AR head-wearable display, a VR head-wearable display or a goggles, for example a skiing-goggles, a swimming-goggles, a sports helmet, a head-band structure or a welding protective googles.

In embodiments referring to spectacle devices, the frame may include a first frame portion at least partly surrounding a first ocular opening, a second frame portion at least partly surrounding a second ocular opening, and the central portion may form (or be formed by) a bridge portion mechanically connecting the first frame portion and the second frame portion.

In other words, the head-wearable device may include a frame including a first frame portion at least partly surrounding a first ocular opening, a second frame portion at least partly surrounding a second ocular opening, a bridge portion mechanically connecting the first frame portion and the second frame portion, and a first electric connector arranged on and/or in the bridge portion.

The frame of the head-wearable device may include a passage opening for receiving a protruding portion of the scene camera of the camera module as explained herein.

According to an embodiment of a method for manufacturing a camera module for a head-wearable device, the method includes providing a camera connecting structure equipped with a first eye camera and a second eye camera, providing a mold comprising a first fixation structure for the first eye camera and a second fixation structure for the second eye camera, arranging the connecting structure in the mold so that the first eye camera is arranged in and/or at the first fixation structure, and the second eye camera is arranged in and/or at the second fixation structure, filling the mold with a material, and solidifying the material to at least substantially fix a positional relationship and a orientational relationship of the first eye camera and the second eye camera with respect to each other.

Accordingly, the (relative) camera poses of the camera module may be set precisely defined in the mold, and long-term stably be fixed by curing the material filled into the mold.

Solidifying may include and/or be achieved by curing of the material, in particular chemical or thermal curing.

As already explained above, the camera connecting structure itself is typically at least partly flexible to facilitate arranging the cameras in the mold in the desired positions and orientations temporarily (during molding) fixed by respective fixation structures.

The camera connecting structure may in particular include an at least partly flexible board such as a flex PCB or a rigid-flex PCB.

The camera connecting structure is typically additionally provided with a scene camera.

However, with the molding process described herein, it is also possible to manufacture a camera module without a scene camera.

Likewise, it is also to manufacture a camera module without a scene camera and one eye camera only with the molding process described herein.

Further, the camera connecting structure may be provided with a first light source for illuminating a field of view of the first eye camera, and/or a second light source for illuminating a field of view of the second eye camera, in particular with a respective (infrared) LED.

The mold may include a third fixation structure for the scene camera and/or a recess for partly accommodating the scene camera in the mold.

Accordingly, arranging the connecting structure in the mold may include arranging the scene camera in and/or at the third fixation structure of the mold, in the recess and/or on a sealing support of the mold for the scene camera, in particular on and/or at a respective sealing support for an optical entrance of the scene camera.

Separate fixations structures for the optional (infrared) light sources are typically not required.

However, if the poses of those light sources play a role in determining gaze-related parameters, as for example in glint-based eye tracking systems, the same manufacturing strategies as described herein using fixation receptacles for eye cameras can be used for (infrared) light sources.

The first fixation structure is typically implemented as a first receptacle for a portion of the first eye camera.

Likewise, the second fixation structure is typically implemented as a second receptacle for a portion of the second eye camera.

The first receptacle may at least substantially positively fit with a housing of the first eye camera next to an optical entrance of the first eye camera.

Likewise, the second fixation structure may at least substantially positively fit with a housing of the second eye camera next to an optical entrance of the second eye camera.

Typically, the first fixation structure and/or the second fixation structure are implemented as a respective recess or hole in a wall of the mold.

The connecting structure is typically arranged in the mold so that the cameras' poses are fixed in the mold, for example so that a portion of the first eye camera is arranged in the first receptacle and a portion of the second eye camera is arranged in the second receptacle.

The material used for molding is typically, at least in the solidified state, transparent.

This facilitate manufacturing a watertight camera module with the eye cameras (as well as the optional light sources) being substantially or even completely embedded. Note that even the respective optical camera entrance may be covered with a thin layer of the solidified transparent material.

The material may be a plastic or a silicone.

The manufactured camera module is typically shaped to partly surround a nose of the user but not shaped as a nose bridge portion but rather spaced apart from the user's face, in particular the user's nose when wearing the head-wearable eye tracking device, and/or at least substantially mirror symmetric with respect to a (virtual, central) symmetry plane.

In the following, the symmetry plane is also referred to as vertical plane.

The term "at least substantially mirror symmetric with respect to a plane" as used within specification intends to describe that at least the outer surface or boundaries of two elements or objects are mirror symmetric with respect to the plane with an accuracy which is typically achievable with the materials and manufacturing methods used, in particular with an accuracy of at least <NUM> % or even at least <NUM> % or <NUM>%, at least on average, and/or that two (virtual) parts of an outer surface or boundary of one element or object are mirror symmetric with respect to the plane with an accuracy of at least <NUM>% or even at least <NUM> % or <NUM>%, at least on average. In particular, a centre of gravity of the outer surfaces lies within the plane or is at most about <NUM> or even at most about <NUM>, more typically at most about <NUM> spaced apart from the plane.

The term that "a second distance at least substantially corresponds to the first distance" as used within specification intends to describe that a relative difference between the first distance and the second distance is at most about <NUM>%, more typically at most about <NUM> % or even <NUM>%.

The components in the figures are not necessarily to scale, instead emphasis being placed upon illustrating the principles of the invention. Moreover, in the figures, like reference numerals designate corresponding parts. In the drawings:.

For sake of clarity, the Figures are provided with a respective Cartesian coordinate system x, y, z typically representing a respective eye tracking module coordinate system and scene camera coordinate system, respectively.

In the following Detailed Description, reference is made to the accompanying drawings, which form a part hereof, and in which is shown by way of illustration specific embodiments in which the invention may be practiced. In this regard, directional terminology, such as "top," "bottom," "front," "back," "leading," "trailing," etc., is used with reference to the orientation of the Figure(s) being described. Because components of embodiments can be positioned in a number of different orientations, the directional terminology is used for purposes of illustration and is in no way limiting. It is to be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present invention. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.

Points and directions may be specified within the scene camera images, an eye camera coordinate system, a scene camera coordinate system, an eye tracking module coordinate system, a head coordinate system, a world coordinate system and/or any other suitable coordinate system.

With reference to <FIG>, an embodiment of a camera module <NUM> for complementing a support structure of an electronic head-wearable device of a user is explained.

As shown in <FIG> illustrating a front view of camera module <NUM>, a one-piece base structure <NUM> of module <NUM> may be mirror symmetric with respect to a virtual central symmetry plane S. Further, a lower portion <NUM> of base structure <NUM> may have a left leg portion <NUM> and a right portion leg portion <NUM> formed such that a recess <NUM> for a user's nose is formed in lower portion <NUM>.

In the exemplary embodiment, one-piece base structure <NUM> accommodates in an upper portion <NUM> a scene camera <NUM> arranged in plane S, and a two-part electric connector <NUM>, <NUM>, which is typically implemented as a two-part electro-mechanical connector, and, in the lower portion <NUM>, a left (first) eye camera <NUM> and a right (second) eye camera <NUM>.

More particular, the left leg portion <NUM> accommodates the left eye camera <NUM>, and the right leg portion <NUM> accommodates the right eye camera <NUM>.

The left eye camera <NUM> and the right eye camera <NUM> are arranged mirror symmetric to each other with respect to central plane S. Further, both the scene camera <NUM> and the two parts of the electric connector <NUM>, <NUM> are mirror symmetric with respect to central plane S.

In front view and in a projection onto one or even two (orthogonal) planes P, M perpendicular to the symmetry plane S and the optical axis <NUM> (see also <FIG> illustrating a top view of module <NUM>), respectively, the scene camera <NUM> is arranged between the two parts of electric connector <NUM>, <NUM>. Typically, their centres lie on a line in the respective view/projection. However, the optical axis <NUM> may also be tilted with respect to plane M, for example by up to <NUM>° or even <NUM>°.

For the sake of clarity, not the eye cameras <NUM>, <NUM> but recesses <NUM>, <NUM> in one-piece base structure <NUM> for accommodating the eye cameras <NUM>, <NUM> are illustrated in <FIG>. For the same reason, neither electrical or mechanical contacts typically arranged inside one-piece structure <NUM> are shown in <FIG> nor is the electric connector <NUM>, <NUM> shown in <FIG>. In <FIG>, eye cameras <NUM>, <NUM> are indicated as slightly protruding from the base structure <NUM>, however they can be embodied completely integrated into the base structure, with no protruding parts.

A first distance between the (first) left eye camera <NUM> and the symmetry plane S is typically at most <NUM>, at least in front view and in the projection onto the plane P, respectively. Likewise, a second distance between the (second) right eye camera <NUM> and the symmetry plane S is typically at most <NUM>, at least in front view and in the projection onto the plane P, respectively. In the exemplary embodiment, the eye cameras <NUM>, <NUM> are mirror-symmetric with respect to symmetry plane S. Accordingly, the first distance matches the second distance. The first and second distances may be respective minimum distances (each dy1/<NUM> in the exemplary embodiment) or more typically respective maximum distances (each dy2/<NUM> in the exemplary embodiment). Said first and second distances may also measure the respective distance between an (optical) center of the eye camera <NUM>, <NUM> and the symmetry plane S, in the direction perpendicular to plane S. Further, said camera centers may for example be halfway between dy1/<NUM> and dy2/<NUM>.

In a projection onto a plane M which is perpendicular to the symmetry plane S and the plane P and in top view, respectively, scene camera <NUM> is spaced apart from the parts <NUM>, <NUM> of electric connector <NUM>, <NUM>.

As illustrated in <FIG>, a first angle α<NUM> between an optical axis <NUM> of left eye camera <NUM> and symmetry plane S, and a second angle α<NUM> between an optical axis <NUM> of right eye camera <NUM> and symmetry plane S may be about <NUM>° (or -<NUM>°).

Furthermore, the first angle α<NUM> may corresponds to the second angle α<NUM>.

As illustrated in <FIG>, a second angle <NUM>°-β<NUM> between optical axis <NUM> of left eye camera <NUM> and a plane M' (or plane M), which is parallel to the x-y plane as indicated by the coordinate systems and perpendicular to symmetry plane S, and a fourth angle <NUM>°-β<NUM> between optical axis <NUM> of right eye camera <NUM> and plane M' (or plane M) may be both about <NUM>° (or -<NUM>°), and/or at least substantially equal.

The total height dz of one-piece base structure <NUM> (extension of modular base structure <NUM> in a direction perpendicular to plane M, in z-direction) may be in a range from about <NUM> to about <NUM>, more typically in a range from about <NUM> to about <NUM>.

The widths of one-piece base structure <NUM>, i.e. the extension of base structure <NUM> in a direction perpendicular to the symmetry plane S typically varies as function of the height (z) and is typically in a range from about <NUM> to <NUM>, more typically in a range from about <NUM> to about <NUM>.

As illustrated in <FIG>, the width dy<NUM> of upper portion <NUM> may be substantially constant and/or lower than the maximum width dy<NUM> of lower portion <NUM>, for example by a factor of <NUM> to <NUM>.

As illustrated in <FIG>, legs (leg portions) <NUM>, <NUM> are typically bent.

Alternatively or in addition, the depth dx1 (extension in direction of optical axis <NUM> of scene camera <NUM>, which in the example shown is parallel to the x-direction) of one-piece base structure <NUM> may vary as function of height.

The depth dx1 may be in a range from about <NUM> to about <NUM>, more typically in a range from about <NUM> to <NUM>.

The total extension dx in x-direction may be in a range from about <NUM> to about <NUM>, more typically in a range from about <NUM> to <NUM>.

As illustrated in <FIG>, the scene camera <NUM> may protrude from one-piece base structure <NUM>.

However, the scene camera <NUM> and/or the electric connector <NUM>, <NUM> may also be arranged within one-piece base structure <NUM>, i.e. such that they are accessible from the outside but do not protrude from the base structure.

As illustrated in <FIG>, a camera connecting structure <NUM> for electrically connecting scene camera <NUM> and eye cameras <NUM>, <NUM> may be embedded in one-piece base structure <NUM> and connected with the cameras <NUM>, <NUM>, <NUM>.

<FIG> illustrates a front view of a camera module <NUM> for attachably complementing a support structure of a head-wearable device of a user.

Camera module <NUM> is typically similar to camera module <NUM> explained above with regard to <FIG> and also has substantially transparent one-piece base structure <NUM> with a lower portion <NUM> with two leg portions <NUM>, <NUM>, but is illustrated in more detail.

This applies in particular to the exemplary two-part electric-mechanical connector <NUM>, <NUM> formed in upper portion <NUM> left and right of scene camera <NUM> as respective socket with three electrical contacts <NUM> (shown in <FIG>, 3C, 3E in different views) for typically spring-loaded pins <NUM> of a carrier <NUM> of a head-wearable device <NUM> (shown in <FIG>) to be complemented with camera module <NUM>.

While <FIG> is also a front view of camera module <NUM> including carrier <NUM> hiding connector <NUM>, <NUM>, <FIG> shows a cross-sectional view along line A in <FIG> is a back view, and <FIG> is a side view of camera module <NUM> and a carrier <NUM>, respectively.

As indicated in <FIG> by elements <NUM> that may e.g. represent holes or screws, carrier <NUM> may be (typically permanently) fastened with the central portion <NUM> of the support structure by two fastening means arranged mirror-symmetric with respect to symmetry plane S, typically close to symmetry plane S, e.g. at a distance of at most <NUM> or even at most <NUM>, and/or above and/or close to the scene camera <NUM>. This allows for a particularly stable and mechanically decoupled fastening of camera module <NUM> to the support structure of head-wearable device <NUM>.

Alternatively, or in addition, an adhesive (glued) joint may be formed between carrier <NUM> and central portion <NUM>.

As indicated by reference numerals <NUM>, carrier <NUM> and camera module <NUM> may be connected via a screw connection.

<FIG> illustrates a back view of a camera connecting structure 370a-f which will be embedded in one-piece base structure <NUM> of camera module <NUM> in the course of the manufacturing process according to embodiments.

In the exemplary embodiment, camera connecting structure 370a-f includes in lower portion <NUM> a circuit board 210a equipped with left eye camera <NUM>, a circuit board 210d equipped with right eye camera <NUM>, and a central circuit board 270b with a central through hole for the scene camera (objective) arranged centrally between the six electrical contacts (<NUM>).

Each circuit board 210a, 270d is, in a respective region 270f, flexibly (bendably) connected with circuit board 270b.

Further, circuit board 270b is connected with a further circuit board 270c arranged in upper portion <NUM> and equipped with one or more components such as an IC implementing a controller <NUM> of camera module <NUM>.

Controller <NUM> may be functionally connected with cameras <NUM>, <NUM>, <NUM>, in particular in electric connection with cameras <NUM>, <NUM>, <NUM>.

Controller <NUM> may be configured to control cameras <NUM>, <NUM>, <NUM>.

Further, controller <NUM> may be configured to process images received from cameras <NUM>, <NUM>, <NUM>.

For example, controller <NUM> may be configured to calculate one or more gaze-related parameters of a user based on the received images.

<FIG> illustrates a perspective view of a head-wearable device <NUM> implemented as spectacle device, and the camera module <NUM> as shown in <FIG> that may be attached to head-wearable device <NUM> to form a head-wearable eye tracking device <NUM>.

In the exemplary embodiment, head-wearable device <NUM> is implemented as spectacles device. Accordingly, a frame <NUM> of spectacles device <NUM> forms a support structure of spectacles device <NUM>. Frame <NUM> has a front portion <NUM> only partly surrounding a left ocular opening and a right ocular opening. A central portion <NUM> of front portion <NUM> forms a bridge portion arranged between the ocular openings and spectacles lenses, respectively. Further, a left temple <NUM> and a right temple <NUM> are attached to front portion <NUM>.

Furthermore, a passage opening for the (hidden) scene camera of eye tracking module <NUM> and the field of view (FOV) of the scene camera, respectively, is formed in bridge portion <NUM>.

Likewise, carrier <NUM> accommodated in bridge portion (central portion) <NUM> and arranged on the wearer-side of bridge portion <NUM> has a through hole for the scene camera and six exemplary spring-loaded pins of two plugs for (fitting with) the two-part (first) electric socket (connector) <NUM>, <NUM> shown in <FIG>.

Eye tracking module <NUM> is shaped at least substantially complementary to central (bridge) portion <NUM> of frame <NUM> of head-wearable device <NUM>, more particular to fit into a recess or opening <NUM> of central portion <NUM> with attached carrier <NUM> of head-wearable device <NUM> (see also <FIG>).

In a lower part <NUM>, central portion <NUM> of frame <NUM> is formed as a nose-bridge portion. Accordingly, lower part <NUM> has appropriately shaped contact surface 603c for the user's nose and the bridge of the user's nose, respectively. Contact surface 603c may also be considered as support surface of central portion <NUM> and frame <NUM>, respectively.

A first nose pad and a second pad (not shown) may be attached to lower part <NUM>, but typically not to leg portions (camera arms) <NUM>, <NUM>. The first nose pad and/or the second nose pad may be adjustable and/or exchangeable attached to the lower part <NUM>.

A second electric connector (not shown) in electric contact with the first two-part electric connector of carrier <NUM> (via a not shown internal wiring) may be accommodated at the dorsal and of left or right temple <NUM>, <NUM>.

The second electric connector may be used for connecting with a companion device such as a smart phone or tablet. For example, the second electric connector may be a USB-connector.

<FIG> illustrates a perspective view of head-wearable eye tracking device <NUM> including camera module <NUM> in assembled state. In the exemplary embodiment, both the first leg portion <NUM> and the second leg portion <NUM> are spaced apart from central portion <NUM> of frame <NUM> (and the user wearing eye tracking device <NUM>). Accordingly, the (relative) camera poses remain at least substantially fixed under normal wearing/use conditions. And are at least largely independent of any deformation of frame <NUM> under these conditions, respectively.

As illustrated in <FIG>, symmetry plane S of module <NUM> may, in the mounted state, also represent a symmetry plane for frame <NUM>.

<FIG> illustrates, in an exploded top view, two bending states of head-wearable eye tracking device <NUM> shown in <FIG>.

Due to the central fixation of camera module to the frame <NUM>, optionally via carrier <NUM>, the angle α<NUM> between and an optical axis <NUM> of left eye camera <NUM> and symmetry plane S, and a second angle α<NUM> between an optical axis <NUM> of right eye camera <NUM> and symmetry plane S, and, thus, the relative angle α<NUM> + α<NUM> between optical axes <NUM>, <NUM> is substantially independent of flex of the head-wearable indicated by distance dy, dy<NUM> between the ends of the temples <NUM>, <NUM>, at least for typical distances dy, dy<NUM>.

Different thereto, the corresponding angles typically vary substantially depending on the bending state of a head-wearable eye tracking devices <NUM> with integrated eye cameras as illustrated in <FIG> by the angle ranges Δα<NUM>, Δα<NUM>. This may have a substantial impact on accuracy of eye parameter determination.

In the illustrated embodiment, carrier <NUM> is fastened to central portion <NUM> with two screws in a mirror-symmetric way with respect to plane S and at a respective distance df/<NUM> of <NUM> from plane S.

With regard to <FIG> methods for manufacturing a camera module and the manufactured camera module, respectively, are explained.

As shown in the flow chart of <FIG>, a method <NUM> for manufacturing a camera module for complementing a support structure of a head-wearable device, in particular a camera module as explained herein, method <NUM> may include providing a camera connecting structure assembled with a first eye camera, and a second eye camera, in a block <NUM>, and providing a mold with a first fixation structure for the first eye camera and a second fixation structure for the second eye camera, in a block <NUM>.

Thereafter, the connecting structure may be arranged in the mold so that the first eye camera is stably held by the first fixation structure, and the second eye camera is stably held by the second fixation structure, in a block <NUM>.

Thereafter, the mold is, in a block <NUM>, filled with a liquid material that can be solidified to from a long-term stable, substantially transparent and mechanically sufficiently stable material, in particular a respective silicone or plastic.

Thereafter, the material is solidified in a block <NUM>.

Solidifying typically includes curing of the material, in particular chemical or thermal curing, and/or may be a two-stage process with a prolonged post-cure process.

Accordingly, a positional and orientational relationship of the first eye camera and the second eye camera with respect to each other may be at least substantially fixed and reproducibly obtained.

Thereafter, the manufactured camera module may be removed from the mold in a block <NUM>, and method <NUM> may return to block <NUM> as indicated by the dashed-dotted arrow in <FIG>.

Thereafter, the manufactured camera module may be connected with a central portion of a support structure of a head-wearable eye device, typically via a plug-socket connection and/or a carrier arranged in between.

Typically, the camera connecting structure is additionally provided with a scene camera which is also stably held in the mold during filling and solidifying.

<FIG> shows a side view of an exemplary two-part mold 400a, 400b and a camera connecting structure 370a, 370b, 370c, 370f which is additionally provided with a scene camera <NUM>, and which is arranged in mold 400a, 400b with its parts 400a, 400b arranged one above the other.

In the schematic side view of <FIG>, the typically provided second eye camera is hidden by the first eye camera <NUM>. This also applies to <FIG> showing a corresponding side view after filling the voids <NUM>' in mold 400a, 400b with a material, solidifying the material, and opening mold 400a, 400b to remove the manufactured camera module <NUM> with the camera connecting structure 370a-f being embedded in a one-piece base structure <NUM> formed in the mold (in the mold's void <NUM>').

In the exemplary embodiment, camera connecting structure 370a, 370b, 370c, 370f is flexible in a connecting part or region 310f between a part 370a equipped with first eye camera <NUM> and a part 370b equipped with scene camera <NUM> (and likewise between a part 370b and a hidden part equipped with the second eye camera), but may be sufficiently stiff in the other parts, in particular part <NUM>.

Accordingly, camera connecting structure 370a, 370b, 370c, 370f may be placed in lower part 400a which is followed by covering lower part 400a with upper part 400b, and optionally covering the optical entrance of scene camera <NUM> with a plug 400c if the objective of scene camera <NUM> is desired to be laterally covered by the material but not on top (at the lens), so that the camera poses are held during filling and curing.

This way, manufacturing tolerances and imperfections of the typically massproduced camera connecting structures 370a-f do not negatively impact the reproducibility of the camera poses of the final camera module, since all modules are produced using the same (single) highly accurate mold that tightly grips the cameras and thus precisely defines and reproduces their mutual poses.

In the exemplary embodiment, eye camera <NUM> is held by a void <NUM> in lower part 400a fitting with eye camera <NUM> next to the optical entrance of eye camera <NUM> and a protrusion 400d of the upper part 400b.

In the closed state of mold 400a, 400b shown in <FIG>, void <NUM> and protrusion 400d form a fixation structure for eye camera <NUM>.

Void <NUM> and protrusion 400d may be centred with respect to each other and an optical axis <NUM> of eye camera <NUM>, respectively.

Further, in the exemplary closed state of mold 400a, 400b, scene camera <NUM> is not held by void <NUM> of upper part 400b and a corresponding support structure of lower part 400a, which is also possible and generally preferred, but by a fixation structure 400e, 400f for the rigid parts 370a, 370c, e.g. a common PCB board 370b, 370c for the scene camera <NUM> and a controller of camera module <NUM>.

In the exemplary embodiment, fixation structure 400e, 400f is formed by plugs that fit into a hole of common PCB board 370b, 370c from top and bottom, respectively.

While the manufacturing method is typically used for manufacturing camera modules with three cameras, it may also be used for manufacturing a camera module with only two cameras, i.e. with two eye cameras, or with one eye camera and one scene camera.

According to an embodiment a method for manufacturing a watertight camera module for complementing a head-wearable device, the method includes: providing a camera connecting structure equipped with at least two of, typically all of: a first eye camera for taking first images of at least a portion of a first eye of a user wearing the head-wearable eye tracking device formed by complementing the head-wearable device with the camera module, a second eye camera for taking second images of at least a portion of a second eye of the user wearing the head-wearable eye tracking device, and a scene camera for taking scene images of a visual field of the user wearing the head-wearable eye tracking device, and optionally a first light source for illuminating the first eye and/or a second light source for illuminating the second eye, in particular a respective (infrared) LED. The method further includes providing a mold, arranging the connecting structure in the mold so that the camera (s) is (are) kept stable (fixed) in terms of location and orientation in the mold, filling the mold with a material; and solidifying the material.

In a closed state, the mold may provide a respective fixation structure for the camera(s) and/or a rigid part of the camera connecting structure next to the respective camera.

Although various exemplary embodiments of the invention have been disclosed, it will be apparent to those skilled in the art that various changes and modifications can be made which will achieve some of the advantages of the invention without departing from the spirit and scope of the invention. It will be obvious to those reasonably skilled in the art that other components performing the same functions may be suitably substituted. It should be mentioned that features explained with reference to a specific figure may be combined with features of other figures, even in those cases in which this has not explicitly been mentioned. Such modifications to the inventive concept are intended to be covered by the appended claims.

Spatially relative terms such as "under", "below", "lower", "over", "upper" and the like are used for ease of description to explain the positioning of one element relative to a second element. These terms are intended to encompass different orientations of the device in addition to different orientations than those depicted in the figures. Further, terms such as "first", "second", and the like, are also used to describe various elements, regions, sections, etc. and are also not intended to be limiting. Like terms refer to like elements throughout the description.

Claim 1:
A camera module (<NUM>, <NUM>, <NUM>) for complementing a support structure of a head-wearable device (<NUM>), in particular a respective frame (<NUM>, <NUM>, <NUM>) of the head-wearable device, to form a head-wearable eye tracking device (<NUM>, <NUM>, <NUM>), the camera module (<NUM>, <NUM>, <NUM>) comprising:
- a one-piece base structure (<NUM>, <NUM>, <NUM>) comprising an at least semi-transparent material, typically a substantially transparent material, and comprising an upper portion (<NUM>, <NUM>, <NUM>), a first leg portion (<NUM>, <NUM>) and a second leg portion (<NUM>, <NUM>);
- a camera connecting structure (<NUM>, 270a, 270b, 270c, 270d, 270f, 270f', 370a, 370b, 370c, 370f) embedded in the one-piece base structure (<NUM>, <NUM>, <NUM>);
- a first eye camera (<NUM>, <NUM>, <NUM>) connected with the camera connecting structure (<NUM>, 270a, 270b, 270c, 270d, 270f, 270f', 370a, 370b, 370c, 370f), and arranged in and/or on the first leg portion (<NUM>, <NUM>);
- a second eye camera (<NUM>, <NUM>) connected with the camera connecting structure (<NUM>, 270a, 270b, 270c, 270d, 270f, 270f', 370a, 370b, 370c, 370f), and arranged in and/or on the second leg portion (<NUM>, <NUM>); and
- a scene camera (<NUM>, <NUM>, <NUM>) connected with the camera connecting structure (<NUM>, 270a, 270b, 270c, 270d, 270f, 270f', 370a, 370b, 370c, 370f), and arranged in and/or on the upper portion (<NUM>, <NUM>, <NUM>).