STEERABLE CAMERA ARRAY FOR HEAD-MOUNTED DISPLAY DEVICES

A head wearable apparatus, such as a pair of smart glasses are configured to track a gaze direction of a person for various applications. To support the tracking, the head wearable apparatus is configured with a lens assembly including at least one lens module (e.g., a lens or a lenslet array) operably coupled to the head wearable apparatus and a camera assembly including a camera sensor operably coupled to the head wearable apparatus. The head wearable apparatus is also configured with a camera assembly operably coupled to an anterior surface of a flexure of the head wearable apparatus (e.g., proximate to the front of the smart glasses frames) to minimize a size (e.g., to be as small as possible) of an aperture housing the camera assembly. The flexure is configured to be adjacent to the lens assembly operably coupled to the head wearable apparatus.

BACKGROUND

The use of head mounted display (HMD) devices, such as smartglasses, continues to increase at a rapid pace. For instance, head wearable apparatuses have increasingly become an integral part of the way in which users interact with different computer applications, such as virtual reality (VR) applications, augmented reality (AR) applications, gaming applications, among other examples. Typically, these HMD devices are configured with a small display system (e.g., an optic in front of one (monocular HMD) or each eye (binocular HMD)), a camera, and other electronic components operably configured with the display system and the camera to support various operations. However, existing configurations of these components can require large or bulky HMD form factors, or can cause poor camera performance, resulting in a poor user experience. Some wearable devices, such as smart eyeglasses, have insufficient space to support both a display system and a camera being positioned side-by-side in the same physical region of the smart eyeglasses (e.g., on the same temple (also referred to as arm) of the smart eyeglasses, or on the same frame front of the smart eyeglasses and still meet an acceptable form factor). In order to support binocular eyeglasses (i.e., a display in both eyes), it is desirable for the display system and the camera to coexist in the same region of the smart eyeglasses.

DETAILED DESCRIPTION

Typically, a head wearable apparatus (or device), such as smartglasses are configured with a small display system (e.g., an optic in front of one (monocular HMD) or each eye (binocular HMD)), a camera system, and other electronic components operably configured with the display system and the camera to support various operations. The available volume adjacent to the display system for the camera system is significantly limited by the desired form factor of the smartglasses. However, the central gaze direction (also referred to as field-of-view) of the user is typically 5 degrees wide, and the para-central gaze direction is typically 8 degrees wide. A user will turn their eyes within a range of about +/−15 degrees to view an object. When an object of interest falls outside of this range, the user might typically turn their head to see the object clearly. It may be desirable to have a camera assembly configured with the head wearable apparatus that might be limited to having a field-of-view comparable to the central or paracentral region of the user, to be capable to monitor the gaze direction of the user and adjust the pointing direction of the camera assembly so that the gaze direction of the user is within the field-of-view of the camera assembly.

Various aspects of the present disclosure relate to a head wearable apparatus (e.g., smartglasses) configured with a display system (also referred to as display interface) and a camera coexisting on the same side of the head wearable apparatus. For example, the wearable apparatus is configured with a lens assembly including at least one lens module (e.g., at least one lens or a lenslet array) operably coupled to the head wearable apparatus and a camera assembly including at least one sensor (e.g., a camera sensor, among other examples) operably coupled to the head wearable apparatus. In some embodiments, the head wearable apparatus may be configured with a sensory array (e.g., up to five sensors, with each sensor being an array of 400×400 pixels or 400×300 pixels, etc.). The head wearable apparatus is also configured with a camera assembly operably coupled to an anterior surface of a flexure of the head wearable apparatus (e.g., proximate to the front of the smart eyeglasses frames) to minimize a size of an aperture housing the camera assembly (also referred to as camera assembly housing) and the optical aperture window. The flexure is configured to be adjacent to the lens assembly operably coupled to the head wearable apparatus and control rotation of the camera assembly via its axis. As such, the direction of scanning of the camera array may be horizontal. In other words, the flexure allows rotation of the camera array to track the side-to-side movement of the user's eyes.

FIG.1illustrates a wearable device100in accordance with some embodiments. The wearable device100may be an example of a pair of smart eyeglasses configured to track a gaze direction of a person wearing the smart eyeglasses for various applications, such as streaming applications, gaming applications, among other examples. In some other embodiments, the wearable device100is another head-wearable device, such as a head-mounted display (HMD) device that has a display optic in front of one (monocular HMD) or each eye (binocular HMD).

The device100may be worn on a user's head and, when so worn, secures at least one electronic display within a viewable field of at least one of the user's eyes, regardless of the position or orientation of the user's head. The device100as described herein may include a display system that enables a user to see displayed content but also does not prevent the user from being able to see their external physical environment. The display system may be either transparent or at the periphery of the user's field-of-view (gaze direction), so that it does not completely block the user from being able to see their external physical environment.

As illustrated inFIG.1, the wearable device100includes a frame102, which may be composed of plastic and acetate or other materials. These materials can be used to produce frames of all colors, shapes, patterns, and even textures while maintaining a comfortable lightweight. In some other embodiments, the frame102may be composed of a metal material. This material can be used for rimless and semi-rimless frame styles and support a variety of shapes, colors, and styles. The frame102includes a pair of temples104(also referred to as arms or holders) that extend in a first direction along a first axis when in a first configuration (e.g., folded) and along a second axis when in a second configuration (e.g., unfolded). In some embodiments, the pair of temples includes a first temple104-1and a second temple104-2that are parallel to each other and are perpendicular to the frame102when in an unfolded configuration. The frame102, in some embodiments, includes a pair of shoulders120including a first shoulder120-1which span a region between the lens108-1and the temple104-1, and a second should120-2which may span a region between the lens108-2and the temple104-2.

In some embodiments, the frame102includes a pair of lens holders106including a first lens holder106-1and a second lens holder106-2. A pair of lenses108are operably coupled to the pair lens holders. For example, a first lens108-1is operably coupled to a first lens holder106-1and a second lens108-2is operably coupled to a second lens holder106-2. The frame102includes a foreframe110including a bridge that is the center of the frame102and that rests on a person's nose and joins the lens holders106together.

The frame102includes a pair of flexures112including a first flexure112-1and a second flexure112-2. The flexures112may be curved or bent to fit the frame102configuration. In some embodiments, the pair of flexures112are proximate to the foreframe110. In some other embodiments, the pair of flexures112are proximate to the pair of temples104(also referred to as arms holders). The pair of flexures112may be shaped to fit the configuration of the frame102. It may be attached to the foreframe110and at least one temple104of the pair of temples104by one or more means. Although the wearable device100illustrates multiple components, the present disclosure applies to any wearable device100architecture having more or fewer components.

The frame102may include an “eyeward” side that faces the user's eyes when the frames are worn and an “outward” side that is opposite the eyeward side. In the example ofFIG.1, at least one shoulder120may include a lens module coupled to at least one surface of the frame102and a camera assembly114operably coupled to an anterior surface of a flexure112(e.g., the flexure112-1) on the outward side of the frame102. That is, the anterior surface of the flexure112may be part of the flexure112(body) that faces forward. The flexure112-1may be adjacent to the at least one lens108. The camera assembly114may include at least one sensor to track a gaze direction of a user wearing the wearable device100. In some embodiments, the lens module includes at least one lens or a lenslet array. The camera assembly114may be configured to track a position of at least one eye of the user based on a direction of the at least one eye of the user (e.g., a gaze related to movements of the head), or an orientation of the at least one eye of the user, or both. In some embodiments, the wearable device100may be configured with an actuator or other motor component coupled to the camera assembly114and configured to adjust a pointing direction of the at least one sensor (e.g., a camera sensor) or a sensing orientation of the at least one sensor, or both, as described herein.

At least one lens108may be configured with a display interface of a display system that enables a user to see displayed content but also does not prevent the user from being able to see their external physical environment. The display interface may be either transparent or at the periphery of the user's field-of-view (e.g., gaze direction), so that the user is able to see at least part of the surrounding physical environment. In some embodiments, the display interface may be overlaid, for example, positioned on or proximate to a surface of the at least one lens108, and be in parallel with the at least one lens108. The display interface may correspond to an eyebox which may define a range of eye positions over which specific content displayed on via the display interface is visible to the user. The eyebox may be referred to as a volume in space positioned near the display interface of the at least one lens108. When the eye of the user (and more particularly, the pupil of the eye of the user) is positioned inside this volume and facing the display interface of the at least one lens108, the user is able to see all of the content displayed on the display interface. When the eye of the user is positioned outside of this volume, the user is not able to see at least some of the content provided by the display interface.

The camera assembly114may be operably coupled to a controller configured to control an operating mode of the at least one sensor of the camera assembly114. For example, separate camera sensors or sub-regions of camera sensors (e.g. pixel binning) of the camera assembly114may be used to reduce power consumption of the wearable device100by having a single sensor remain active (e.g., always-ON) for sensing purposes, and wake up other sensors when required. In other embodiments, the camera assembly114may employ a single camera sensor or a plurality of the camera sensors perform or assist with various operations (e.g., sensing, monitoring, etc.). For example, a single camera sensor may sense and monitor light waves (as they pass through or reflect off objects) into signals, small bursts of current that convey the information (e.g., used to make an image). The waves can be light or other electromagnetic radiation. A camera sensor may be a charge-coupled device (CCD) or an active-pixel sensor (CMOS sensor), or the like. In some embodiments, a size of at least one camera sensor of the camera assembly114is different from a size of another camera sensor of the camera assembly114. A size of a camera sensor may determine how much light it uses to create an image. A bigger sensor may thus gain more information (e.g., light waves) than a smaller one and produce better images. In some embodiments, pixels associated with a camera sensor may be binned. The camera assembly114may also include an array of camera sensors, where a length (height) of the array of camera sensors is greater than a width of the array of camera sensors as described herein.

In some embodiments, a portion of an anterior surface118(e.g., on an outward side as described herein) of a flexure112includes a mechanical housing116of the camera assembly114and an optical aperture window. An orientation of the camera assembly114may be in a vertical column of the aperture116. The mechanical housing116may include an optical aperture having an opening having an anterior surface and the camera assembly114is positioned in relation to the anterior surface to track the gaze direction of the user. In some embodiments, the anterior surface includes a transparent material. In some other embodiments, a size of the mechanical housing116is based on a size of the portion of the flexure112of the wearable apparatus, for example, in one or more dimensions including direction and orientation (e.g., a vertical direction, a horizontal direction, an angle, etc.). An axis of rotation of the flexure112of the wearable device100is proximate to the anterior surface of the foreframe110. The camera assembly114may be in electrical communication with one or more components of the wearable device100via an electrical interface coupled to the flexure112of the wearable device100. The electrical interface may include a printed circuit board (PCB). The electrical interface may be used to route information (e.g., image data, sensor information) between the camera assembly114and one or more components, as described herein.

FIG.2illustrates a front perspective-view of a flexure and a camera assembly of a wearable device in accordance with some embodiments, andFIG.3illustrates a side perspective-view of the flexure and the camera assembly of the wearable device in accordance with some embodiments. The wearable device includes a pair of flexures202including a first flexure202-1and a second flexure202-2operably coupled to a substrate204. For example, the first flexure202-1may be operably coupled to a first part (e.g., surface) of the camera assembly204, while the second flexure202-2may be operably coupled to a second part (e.g., surface) of the camera assembly204. The camera assembly204may be configured to include an array of camera sensors206including one or more single camera sensor208. The camera assembly204may also include an array of camera sensors, where a length of the array of camera sensors is greater than a width of the array of camera sensors, to reduce a footprint of the camera assembly204. One or more of these components (e.g., the camera assembly204) may be operably adjusted with respect to an axis of rotation.

In some embodiments, the camera assembly204may include 1-4 sensors, and each sensor may have a significant number of pixels (e.g., 400×400 pixels per sensor). For example, the camera assembly204may include an array of N×M camera sensors, where N is a number camera sensors in a vertical direction and M is a number of camera sensors in a horizontal direction. Each camera sensor may have an aspect ratio of P×Q pixels, where P is a number of pixels in a horizontal direction and Q is a number of pixels in a vertical direction. As described herein, N may be greater than M to reduce a footprint of the camera assembly204and to allow the camera assembly204to be configured with the flexure202, which may allow the camera assembly204to coexist on the same side of the wearable device, as a display system as described herein. In some embodiments, N is at least a factor (e.g. multiple) of M, for example, N may be 1.5*M.

FIG.4illustrates a front perspective-view of a flexure and a camera assembly of a wearable device in accordance with some embodiments, andFIG.5illustrates a side perspective-view of the flexure and the camera assembly of the wearable device in accordance with some embodiments. The wearable device may include a minimum number of device electronics (e.g., integrated circuits) in a moveable (adjustable) portion of the wearable device. The wearable device includes a pair of flexures402including a first flexure402-1and a second flexure402-2operably coupled to a camera assembly404. For example, the first flexure402-1may be operably coupled to a first part (e.g., surface) of the camera assembly404, while the second flexure402-2may be operably coupled to a second part (e.g., surface) of the camera assembly404. The camera assembly404may be configured to include an array of camera sensors406including one or more single camera sensors408. One or more of these components (e.g., the camera assembly404) may be operably adjusted with respect to an axis of rotation.

In the examples ofFIGS.4and5, the wearable device includes a connector410(e.g., a flex harness) operably coupled to a PCB412. The moveable portion of the camera assembly404includes the array of camera sensors406including the one or more single camera sensors408. The connector410may provide a circuit path to the PCB412where a subset of the camera assembly404electronics reside. The connector410may be made of flexible-based material, for example, copper, aluminum, nickel, gold, and silver etc. The PCB412may also be used to perform other operations (e.g., image processing, image recognition, tracking) based on information conveyed from the camera assembly404. Thus, in the examples ofFIGS.4and5, the wearable device may include a minimum number of device electronics (e.g., integrated circuits) in a moveable (adjustable) portion of the wearable device by allocating most operations to be performed on the PCB412.

FIG.6illustrates a front perspective-view of a flexure and a camera assembly of a wearable device in accordance with some embodiments, andFIG.7illustrates a side perspective-view of the flexure and the camera assembly of the wearable device in accordance with some embodiments. The wearable device includes a pair of flexures602including a first flexure602-1and a second flexure602-2operably coupled to a camera assembly604. For example, the first flexure602-1may be operably coupled to a first part (e.g., surface) of the camera assembly604, while the second flexure602-2may be operably coupled to a second part (e.g., surface) of the camera assembly604. One or more of these components (e.g., the camera assembly604) may be operably adjusted with respect to an axis of rotation.

The camera assembly604may be configured to include an array of camera sensors606including a first camera sensor606-1, a second camera sensor606-2, a third camera sensor606-3, and a fourth camera sensor606-4. The multiple camera sensors606including the first camera sensor606-1, the second camera sensor606-2, the third camera sensor606-3, and the fourth camera sensor606-4may be coplanar. Although only a single array of camera sensors606is illustrated, the present disclosure may support additional arrays of camera sensors. These camera sensors606may be coupled to an anterior surface of the flexure602as described inFIG.1.

Each camera sensors606may be made up of the same or different sizes (e.g., pixels). The size of each camera sensor606determines how much light it uses to create visual information (e.g., an image). Each camera sensor606consists of a number of light-sensitive spots referred to as photosites, which are used to record and store information about what is seen through a lens of the wearable device. As such, a bigger camera sensor may gain more visual information than a smaller camera sensor and produce better visual information (e.g., images).

In some embodiments, a single separate camera sensor606of the camera assembly604may be used to reduce power consumption of the wearable device by having the single camera sensor606remain active (e.g., always-ON) for sensing purposes, and wake up other camera sensors606when required. For example, the first camera sensor606-1may be configured to remain active for various operations (e.g., tracking a narrow gaze direction), and be configured to trigger (e.g., wake up) the second camera sensor606-2, the third camera sensor606-3, or the fourth camera sensor606-4, or a combination thereof, based on certain operations (e.g., tracking a wide gaze direction).

FIG.8illustrates a side perspective-view of a flexure and a camera assembly of a wearable device in accordance with some embodiments, andFIG.9illustrates a side perspective-view of the flexure and the camera assembly of the wearable device in accordance with some embodiments. The wearable device includes a pair of flexures802including a first flexure802-1and a second flexure802-2operably coupled to a camera assembly804. For example, the first flexure802-1may be operably coupled to a first part (e.g., surface) of the camera assembly804, while the second flexure802-2may be operably coupled to a second part (e.g., surface) of the camera assembly804. One or more of these components (e.g., the camera assembly804) may be operably adjusted with respect to an axis of rotation.

The camera assembly804may be proximate to a lens assembly808-1that is coupled to at least one surface of a frame of the wearable device. In some other embodiments, sensors or a sensory array associated with the camera assembly804may rotate separately from a lens810associated with the lens assembly808-1. In other embodiments, the lens810may rotate with the sensors or the sensory array associated with the camera assembly804. In the example ofFIG.8, the lens assembly808-1may include a single lens808. Alternatively, in the example ofFIG.9, the lens assembly808-1may include a lenslet812including a first lens812-1, a second lens812-2, and a third lens812-3. Thus, as illustrated inFIGS.8and9, either a single lens assembly or a lenslet assembly can be used. The lens810or the lenslet812may have different or varying focal lengths (like an eyeglass progressive lens) so that different regions can be viewed (i.e., distance, intermediate, reading distance, a wide-angle region, etc.). The optical properties of each region may be different (e.g., magnification, MTF etc.) as each region may be used for different experiences on the wearable device. Although the lens810and lenslet812are illustrated as a single optical element, both the lens810and the lenslet812may consist of a number of optical elements, for example, such as in a focusable lens assembly.

FIG.10illustrates a side perspective-view of the flexure and the camera assembly of the wearable device in accordance with some embodiments.FIG.11illustrates a top perspective-view of the flexure and the camera assembly of the wearable device in accordance with some embodiments. The wearable device includes a pair of flexures1002including a first flexure1002-1and a second flexure1002-2operably coupled to a camera assembly1004. For example, the first flexure1002-1may be operably coupled to a first part (e.g., surface) of the camera assembly1004, while the second flexure1002-2may be operably coupled to a second part (e.g., surface) of the camera assembly1004. The camera assembly1004may be configured to include a camera sensor1006. The camera assembly1004may be proximate to a lens assembly1008that is coupled to at least one surface of a frame of the wearable device. The lens assembly1008may include a single lens1008-1. In some embodiments, an aperture1016as described herein, for example, may hold (e.g., house) the camera assembly1004among other components.

In the example ofFIG.10, the wearable device via an actuator (or other motor) may be configured to adjust a direction of the camera assembly1004(e.g., move the camera array side to side, for example, in-and-out), and thereby a pointing direction of a field-of-view of the camera assembly1004as described herein. For example, the wearable device via an actuator (or other motor) may be configured to adjust a pointing direction of the camera assembly1004in order to align a focus plane of the lens1008-1. In some embodiments, the wearable device via an actuator (or other motor) may be configured to adjust a camera sensor (or an array of camera sensors) of the camera assembly1004from a baseline direction1010to an adjusted direction1012, for example, based on tracking a gaze direction of the user wearing the wearable device. In some other embodiments, the wearable device via an actuator (or other motor) may be configured to adjust the camera assembly1004from the baseline direction1010to the adjusted direction1012based on a preconfigured setting (e.g. a direction offset value1014).

FIG.12illustrates a side perspective-view of the flexure and the camera assembly of the wearable device in accordance with some embodiments. The wearable device includes a pair of flexures1202including a first flexure1202-1and a second flexure1202-2operably coupled to a camera assembly1204. For example, the first flexure1202-1may be operably coupled to a first part (e.g., surface) of the camera assembly1204, while the second flexure1202-2may be operably coupled to a second part (e.g., surface) of the camera assembly1204. The camera assembly1204may be configured to include multiple arrays of camera sensors1206including a first array of camera sensors1206-1, a second array of camera sensors1206-2, and a third array of camera sensors1206-3. The camera assembly1204may be proximate to a lens assembly1208that is coupled to at least one surface of a frame of the wearable device. The lens assembly1208may include a single lens1208-1.

In the example ofFIG.12, the wearable device via one or more actuators (or other motor) may be configured to adjust a direction of the camera assembly1204(e.g., move the camera array side-to-side on a corresponding axis, for example, in-and-out), and thereby steer a pointing direction of the field-of-view of the camera assembly1204as described herein. For example, the wearable device via one or more actuators may be configured to adjust a direction of one or more arrays of camera sensors1206including the first array of camera sensors1206-1, the second array of camera sensors1206-2, or the third array of camera sensors1206-3, or a combination thereof, in order to align to the focus plane of the lens1208-1. That is, the camera sensors1206are inclined to a fixed orientation to align to the focal plane of the lens1208-1. The actuator thereby scans the camera sensors1206side-to-side (in/out of the page). In some embodiments, the wearable device via an actuator (or other motor) may be configured to adjust one or more arrays of camera sensors1206of the camera assembly1204. In some other embodiments, the wearable device via an actuator (or other motor) may be configured to adjust the camera assembly1204based on a preconfigured setting. In the example ofFIG.12, one or more of the camera sensors1206may be tilted to align to one or more focal planes of the optics (e.g., the lens1208-1). In some embodiments, this tilt may be a static tilt. That is, the tilt does not get adjusted by an actuator when changing the pointing direction of the one or more camera sensors1206.

FIG.13illustrates a side perspective-view of the flexure and the camera assembly of the wearable device in accordance with some embodiments. The wearable device includes a pair of flexures1302including a first flexure1302-1and a second flexure1302-2operably coupled to a camera assembly1304. For example, the first flexure1302-1may be operably coupled to a first part (e.g., surface) of the camera assembly1304, while the second flexure1302-2may be operably coupled to a second part (e.g., surface) of the camera assembly1304. The camera assembly1304may be configured to include multiple arrays of camera sensors1306including a first array of camera sensors1306-1, a second array of camera sensors1306-2, and a third array of camera sensors1306-3. The camera assembly1304may be proximate to a lenslet assembly1308that is coupled to at least one surface of a frame of the wearable device. The lenslet assembly1310may include multiple single lens1308, for example, a first lens1301-1, a second lens1308-2, and a third lens1308-3.

In the example ofFIG.13, the wearable device via an actuator (or other motor) may be configured to adjust a direction of the camera assembly1304(e.g., move the camera array side-to-side on a corresponding axis, for example, in-and-out), and thereby steer a pointing direction of the field-of-view of the camera assembly1304as described herein. For example, the wearable device via an actuator (or other motor) may be configured to adjust a direction of one or more arrays of camera sensors1306including the first array of camera sensors1306-1, the second array of camera sensors1306-2, and or the third array of camera sensors1306-3, or a combination thereof, in order to align to the focus plane of the lenslet assembly1310. As such, the camera sensors1306are inclined to a fixed orientation to align to the focal plane of the lenslet assembly1310. The actuator thereby scans the camera sensors1306side-to-side on a corresponding axis (in/out of the page). In some embodiments, the wearable device via an actuator (or other motor) may be configured to adjust one or more arrays of camera sensors1306of the camera assembly1304. In some other embodiments, the wearable device via an actuator (or other motor) may be configured to adjust the camera assembly1304based on a preconfigured setting. In the example ofFIG.13, one or more of the camera sensors1306may be tilted to align to one or more focal planes of the optics (e.g., the one or more lenses1308). In some embodiments, this tilt may be a static tilt. That is, the tilt does not get adjusted by an actuator when changing the pointing direction of the one or more camera sensors1306.

FIGS.14A through14Dillustrates perspective-views of the wearable device in accordance with some embodiments. The wearable device includes a flexure1402operably coupled to a camera assembly1404. For example, the flexure1402may be operably coupled to a first part (e.g., surface) of the camera assembly1404. The camera assembly1404may be configured to include a camera sensor1406. The camera assembly1404may be proximate to a lens assembly1408that is coupled to at least one surface of a frame of the wearable device. The lens assembly1408may include a single lens or multiple lenses (e.g., a lenslet).

In the example ofFIG.14, the wearable device may be configured to adjust a direction of the camera assembly1404(e.g., move the camera array side to side, for example, in-and-out), and thereby a pointing direction of a field-of-view of the camera assembly1404as described herein. For example, the wearable device may be configured to adjust a pointing direction of the camera assembly1404in order to align a focus plane of the lens1408. In some embodiments, the wearable device may be configured to adjust a camera sensor (or an array of camera sensors) of the camera assembly1404, for example, based on tracking a gaze direction of the user wearing the wearable device. In some other embodiments, the wearable device may be configured to adjust the camera assembly1404based on a preconfigured setting.

In some embodiments, an aperture1410as described herein, for example, may hold the camera assembly1406. The aperture1410that may include an opening and the camera assembly1406may be positioned to track a gaze direction of a user wearing the wearable device. In some embodiments, an axis of rotation of the aperture (e.g., of the wearable device) is proximate to an anterior surface of the wearable device. For example, an axis of rotation may be as close as possible to the front of a pair of eyeglass frames in order to keep the window aperture as small as possible.

As illustrated inFIGS.14A through14D, the axis of rotation may be different depending on different conditions. As illustrated inFIG.14A, in some embodiments, the axis of rotation is aligned to the aperture1410in order to minimize the aperture1410opening size. In some other embodiments, as illustrated inFIG.14B, the axis of rotation is aligned to the lens1408(e.g., or lens assembly) for optical performance. As illustrated inFIG.14C, in other embodiments, the axis of rotation is aligned to the focal plane (or average focal plane if there are multiple focal planes) of the lens1408(also coincident with the camera sensor(s)106), also for optical performance. Alternatively, as illustrated inFIG.14D, in some embodiments, the axis of rotation is aligned to another location simply because that is where it is easiest to make or mount the components.

FIG.15is a block diagram of a wearable device1502in accordance with some embodiments. The wearable device1502may be an example of a head-mounted display device or other head-wearable devices, as described inFIG.1. The wearable device1502may include components for bi-directional data communications including components for transmitting and receiving communications (e.g., sensor information, visual information), including a lens assembly1504, a camera assembly1506, an actuator1508, an input/output (I/O) controller1510, a transceiver1512, antennas1514, a memory1516, a processor1520, and a modem1522. These components can be in electronic communication via one or more interfaces (e.g., buses, a printed circuit board (PCB)). For example, a flexible electrical connector may connect the camera assembly1506to the rest of the wearable device1502components (e.g., the actuator1508, the I/O controller1510, the transceiver1512, the antennas1514, the memory1516, the processor1520, and the modem1522). Alternatively, a flex PCB may be part of the camera assembly1506, and sensors or sensor boards may exclusively rotate to track a user's gaze wearing the device wearable1502, as described herein.

The lens assembly1504may include at least one lens or a lenslet array. In some embodiments, the lenslet array includes a set of lenslets in a same plane (e.g., x-plane, y-plane, z-plane). Each lenslet, in some embodiments, has the same focal length or different focal lengths, or a combination thereof. The lens assembly1504may be coupled to at least one surface of a frame of the wearable device1502. For example, the wearable device1502may be a pair of wireless or wired eyeglasses having a frame and a pair of temples (also referred to as arms) that extend in a direction perpendicular to the frame of the pair of wireless eyeglasses, for example, when in an unfolded configuration. In some embodiments, the pair of temples extend in a direction parallel to the frame of the pair of wireless or wired eyeglasses, for example, when in a folded configuration. As such, the lens assembly1504may be coupled to at least one temple of the frame of the pair of wireless or wired eyeglasses. It should be understood that other coupling configurations are possible.

The camera assembly1506may be a high-aspect-ratio camera including at least one camera sensor (also referred to as an image sensor). For example, the camera assembly1506may include a single image sensor. In some embodiments, the camera assembly1506includes an array of camera sensors. For example, the camera assembly1506may include an array of image sensors, such as four camera sensors including a number of pixels (e.g., 400×800 pixels). The camera assembly1506, in some other embodiments, includes multiple arrays of camera sensors. In some embodiments, the camera sensors of an array of camera sensors are non-coplanar, for example, to increase an image focus. In some other embodiments, the camera sensors of an array of camera sensors are coplanar, for example, to increase an image focus. The separate camera sensors of the camera assembly1506may also be used to reduce power consumption of the wearable device1502by having a single sensor remain active (e.g., always-ON) for sensing purposes, and wake up other sensors when required. Other functionality use cases may also have a single camera sensor or some of the other camera sensors.

The camera assembly1506may be operably coupled to an anterior surface of a flexure of a frame of the wearable device1502. In some embodiments, the flexure is adjacent to the lens assembly1504. A flexure of the wearable device1502may include a bent or curved portion of an anterior surface (i.e., frontal surface) of the wearable device1502as described inFIG.1. In some other embodiments, the flexure of the wearable device1502may include an edge surface (i.e., temples, arms) of the wearable device1502. The flexure of the wearable device1502may be shaped to fit a configuration of a frame of the wearable device1502. For example, it may be attached to a rim and to a temple piece by any means, such as bolting, pinning, riveting, swaging, threading, bonding by means of adhesives, soldering, and welding.

In some embodiments, a portion of the anterior surface of the flexure includes a mechanical housing that holds the camera assembly1506, as described inFIG.1. The mechanical housing may include an optical aperture that may include an opening having an anterior surface and the camera assembly1506may be positioned in relation to the anterior surface to track a gaze direction of a user wearing the wearable device1502. The anterior surface may include a transparent material. In some embodiments, a size of the aperture is based on a size of the portion of the flexure of the wearable device1502. In some embodiments, an axis of rotation of the flexure of the wearable device1502is proximate to the anterior surface of the wearable device1502. For example, an axis of rotation of the flexure may be as close as possible to the front of a pair of eyeglass frames in order to keep the window aperture as small as possible.

In some embodiments, the camera assembly1506uses either a single lens or a lenslet array of the lens assembly1504to track a gaze direction of a user. Different regions, or facets of the single lens or the lenslet array of the lens assembly1504may have different optical functions. In some embodiments, different regions, or facets of the single lens or the lenslet array of the lens assembly1504have different infinity focus values. In some other embodiments, different regions, or facets of the single lens or the lenslet array of the lens assembly1504have different near focus values. In other embodiments, different regions, or facets of the single lens or the lenslet array of the lens assembly1504have different wide-angle values. In some embodiments, the lens regions are configured to match an optical function of the wearable device1502, for example, smart eyeglasses prescription lenses including progressive lenses.

The camera assembly1506may be configured to provide, in some embodiments, a camera resolution of 400 pixels×3200 pixels. In some embodiments, an orientation of the camera assembly1506may be in a vertical column. A field-of-view width of the camera assembly1506may be 10 degrees (e.g., based on a user's central vision region being nominally 5 degrees wide, and a paracentral vision region being 8 degrees wide). The camera assembly1506may be swept +/−15 degrees, so that the camera assembly1506can monitor, scan, sense, etc. an entire viewing area of the user without head rotation. In some embodiment, beyond +/−15 degrees, the user will turn their head to look at an object.

The actuator1508may be a micro-electronic mechanical system (MEMS) actuator, an open-loop voice coil motor (VCM) actuator, or a closed-loop VCM, among other examples, configured to steer a camera array for HMD devices, such as the wearable device100. In some embodiments, the actuator1508may be coupled to the camera assembly1506and configured to adjust a pointing direction of at least one camera sensor or a sensing orientation of the at least one camera sensor, or both, associated with the camera assembly1506. For example, the camera assembly1506may be mounted on a flexure, and the actuator1508may be used to rotate the camera assembly1506. An eye-tracking system of the wearable device1502may identify where a user is looking and can provide feedback to the wearable device1502(e.g., the camera assembly1506). The camera assembly1506may see through a window (e.g., an aperture) that is integral to the wearable device1502(e.g., glasses frames).

The I/O controller1510can manage input and output signals for the wearable device1502. The I/O controller1510can also manage peripherals not integrated into the wearable device1502. In some embodiments, the I/O controller1510can represent a physical connection or port to an external peripheral. In some other embodiments, the I/O controller1510can utilize an operating system such iOS®, ANDROID®, MS-DOS®, MS-WINDOWS®, OS/2®, UNIX®, LINUX®, or another known operating system. The I/O controller1510can represent or interact with the modem1522(e.g., a 4G modem, a 5G modem), keyboard, a mouse, a touchscreen, or a similar peripheral device. The I/O controller1510can be implemented as part of the processor1520. An end-user can interact with the wearable device1502via the I/O controller1510or via hardware components controlled by the I/O controller1510.

The transceiver1512can communicate bi-directionally, via one or more antennas1514. The transceiver1512can function as a receiver or a transmitter. For example, a receiver and a transmitter can be collocated in the transceiver1512. When operating as a receiver, the transceiver1512can receive information such as packets, control information or user data associated with various information channels (e.g., control channels, data channels, and information related to various applications (e.g., VR applications, AR applications, etc.). Information can be passed on to other components of the wearable device1502. When operating as a transmitter, the transceiver1512can transmit signals generated by other components of the wearable device1502. The wearable device1502can include a single antenna1514or more than one antenna1514, which can be capable of simultaneously transmitting or receiving data communications.

The memory1516can include a random-access memory (RAM) ora read-only memory (ROM). The memory1516can store computer-readable, computer-executable software1518including instructions that, when executed, cause the processor1520to perform various functions described herein. In some embodiments, the memory1516can include, among other components, a BIOS which can control basic hardware or software operation, such as interaction with peripheral components or devices. The software1518can include instructions to implement aspects of the present disclosure, including instructions to support steering a camera sensor or a camera sensor array of the camera assembly1506to track a gaze direction of a user wearing the wearable device1502. The software1518can be stored in a non-transitory computer-readable medium such as system memory or other types of memory. In some embodiments, the software1518cannot be directly executable by the processor1520but can cause the wearable device1502(e.g., when compiled and executed) to perform functions described herein.

The processor1520can include an intelligent hardware device (e.g., a general-purpose processor, a DSP, a central processing unit (CPU), a microcontroller, an ASIC, an FPGA, or other programmable logic device, discrete hardware components, or any combination therefore). In some embodiments, the processor1520can be configured to operate a memory array using a memory controller. In some other embodiments, a memory controller can be integrated into the processor1520. The processor1520can be configured to execute computer-readable instructions stored in the memory1516to cause the wearable device1502to perform various functions (e.g., functions or tasks supporting steering a camera sensor or a camera sensor array of the camera assembly1506to track a gaze direction of a user wearing the wearable device1502).

The modem1522includes radio frequency interfaces configured to support various radio access technologies, for example, 4G LTE and 5G NR. The modem1522can be coupled to the processor1520, the memory1516, the transceiver1512, etc., as described herein. The modem1522can modulate packets and provide the modulated packets to the transceiver1512for transmission. Similarly, the modem1522can receive packets from the transceiver1512and demodulate the received packets from the antennas1514.

The following provides an overview of examples of the present disclosure:

Example 1: A head wearable apparatus, comprising: a lens assembly including a lens module coupled to at least one surface of a frame of the head wearable apparatus; and a camera assembly operably coupled to an anterior surface of a flexure of the frame of the head wearable apparatus, wherein the flexure is adjacent to the lens assembly, the camera assembly including at least one sensor to track a gaze direction of a user.

Example 2: The head wearable apparatus of example 1, wherein the lens module comprises at least one lens or a lenslet array.

Example 3: The head wearable apparatus of example 1 or 2, further comprising: a processor; a memory coupled with the processor; and instructions stored in the memory and executable by the processor to cause the head wearable apparatus to: track a position of at least one eye of the user based on a direction of the at least one eye of the user, or an orientation of the at least one eye of the user, or both.

Example 4: The head wearable apparatus of at least one of the preceding examples, further comprising: an actuator coupled to the camera assembly and configured to adjust a pointing direction of the at least one camera sensor or a sensing orientation of the at least one camera sensor, or both.

Example 5: The head wearable apparatus of at least one of the preceding examples, wherein a portion of the anterior surface of the flexure comprises an aperture housing the camera assembly.

Example 6: The head wearable apparatus of example 5, wherein the aperture comprises an opening having an anterior surface and the camera assembly is positioned in relation to the anterior surface to track the gaze direction of the user.

Example 7: The head wearable apparatus of example 5, wherein the anterior surface comprises a transparent material.

Example 8: The head wearable apparatus of at least one of the preceding examples, wherein a size of the aperture is based on a size of the portion of the flexure of the head wearable apparatus in one or more dimensions including a direction and an orientation.

Example 9: The head wearable apparatus of at least one of the preceding examples, further comprising: a controller configured to control an operating mode of the at least one camera sensor of the camera assembly.

Example 10: The head wearable apparatus of at least one of the preceding examples, wherein an axis of rotation of the flexure of the head wearable apparatus is proximate to: the anterior surface of the head wearable apparatus, the lens assembly of the head wearable apparatus, a focal plane associated with the lens assembly of the head wearable apparatus, or another location of the head wearable apparatus.

Example 11: The head wearable apparatus of at least one of the preceding examples, wherein the camera assembly is in electrical communication with one or more components of the head wearable apparatus via an electrical interface coupled to the flexure of the head wearable apparatus, the one or more components comprising a processor, a memory coupled with the processor, or both, wherein the electrical interface comprises a printed circuit board.

Example 12: The head wearable apparatus of at least one of the preceding examples, wherein the lens assembly is coupled to the frame or the camera assembly.

Example 13: The head wearable apparatus of at least one of the preceding examples, wherein one or more camera sensors of an array of camera sensors of the camera assembly are coplanar.

Example 14: The head wearable apparatus of at least one of the preceding examples, wherein one or more camera sensors of an array of camera sensors of the camera assembly are non-coplanar.

Example 15: The head wearable apparatus of at least one of the preceding examples, wherein a dimension of the at least camera sensor of the camera assembly is different from a dimension of another camera sensor of the camera assembly.

Example 16: The head wearable apparatus of at least one of the preceding examples, wherein the camera assembly comprises: an array of camera sensors, wherein a height of the array of camera sensors is greater than a width of the array of camera sensors, or wherein the width of the array of camera sensors is greater than the height of the array of camera sensors.

Example 17: The head wearable apparatus of at least one of the preceding examples, further comprising: a display system coupled to the lens assembly to output visual information within the gaze direction of the user.

Example 18: The head wearable apparatus of at least one of the preceding examples, further comprising: a head mounted display comprising one or more of the lens assembly and the camera assembly.

Example 19: A pair of wireless-enabled eyeglasses, comprising: a lens assembly including a lens module coupled to at least one surface of a frame of the pair of wireless-enabled eyeglasses; and a camera assembly operably coupled to an anterior surface of a flexure of the frame of the pair of wireless-enabled eyeglasses, wherein the flexure is adjoining to the lens assembly, the camera assembly including a sensor to track a gaze direction of a user wearing the pair of wireless-enabled eyeglasses.

Example 20: The pair of wireless-enabled eyeglasses of example 19, wherein the lens module comprises at least one lens or a lenslet array.