Patent ID: 12221728

In accordance with common practice, the various features illustrated in the drawings may not be drawn to scale. Accordingly, the dimensions of the various features may be arbitrarily expanded or reduced for clarity. In addition, some of the drawings may not depict all of the components of a given system, method, or device. Finally, like reference numerals may be used to denote like features throughout the specification and figures.

BRIEF DESCRIPTION OF THE APPENDIX

Attached to this specification is an Appendix A that includes figures and associated descriptive text for conductive yarns (and knit fabrics formed in part using the conductive yarns or other yarns), forming electrical connections to textile electrodes, and laser cutting certain fabrics (and other manufacturing processes). These aspects can be combined, substituted, or otherwise in conjunction with the other aspects described herein.

DETAILED DESCRIPTION

Numerous details are described herein to provide a thorough understanding of the example embodiments illustrated in the accompanying drawings. However, some embodiments may be practiced without many of the specific details. Furthermore, well-known processes, components, and materials have not necessarily been described in exhaustive detail so as to avoid obscuring pertinent aspects of the embodiments described herein.

Embodiments of this disclosure can include or be implemented in conjunction with various types or embodiments of artificial-reality systems. Artificial reality, as described herein, is any superimposed functionality and or sensory-detectable presentation provided by an artificial-reality system within a user's physical surroundings. Such artificial realities (AR) can include and/or represent virtual reality (VR), augmented reality, mixed artificial reality (MAR), or some combination and/or variation one of these. For example, a user can perform a swiping in-air hand gesture (which can be detected using aspects of the knitted structures described herein) to cause a song to be skipped by a song-providing API providing playback at, for example, a home speaker. In some embodiments of an AR system, ambient light (e.g., a live feed of the surrounding environment that a user would normally see) can be passed through a display element of a respective head-wearable device presenting aspects of the AR system. In some embodiments, ambient light can be passed through respective aspect of the AR system. For example, a visual user interface element (e.g., a notification user interface element) can be presented at the head-wearable device, and an amount of ambient light (e.g., 15-50% of the ambient light) can be passed through the user interface element, such that the user can distinguish at least a portion of the physical environment over which the user interface element is being displayed.

Artificial-reality content can include completely generated content or generated content combined with captured (e.g., real-world) content. The artificial-reality content can include video, audio, haptic events, or some combination thereof, any of which can be presented in a single channel or in multiple channels (such as stereo video that produces a three-dimensional effect to a viewer). Additionally, in some embodiments, artificial reality can also be associated with applications, products, accessories, services, or some combination thereof, which are used, for example, to create content in an artificial reality and/or are otherwise used in (e.g., to perform activities in) an artificial reality.

As described herein, multi-dimensional knitting machines can be leveraged to produce complex knitted structures that include integrating non-knitted structures, adjusting knit patterns and gauges without producing a seam, creating complex garments (e.g., gloves) without requiring the complex garments to be reoriented, etc. While numerous descriptions provided herein reference knitted-fabric structures produced using yarn, the same techniques that are applied to these knitted fabric structures can also be applied to woven-fabric structures.

FIGS.1A-1Eillustrate knitted wearable-glove devices that includes one or more garment-integrated capacitive sensors (e.g., which can be configured to detect force-based and contact-based inputs from fingers of a user118, and can do so in various quadrants for a finer-grain detection of such inputs), in accordance with some embodiments. The knitted wearable-glove device100includes a one or more garment-integrated capacitive sensor assemblies102A-102E in each respective fingertip (and thumb tip (hereinafter finger and fingertip are also used to refer to a thumb and thumb tip)) of the knitted wearable-glove device. Each of the one or more garment-integrated capacitive sensor assemblies102A-102E includes multiple contact regions on each of the respective garment-integrated capacitive sensors (e.g., each respective capacitive sensor includes four distinct contact quadrants, but various other numbers of contact regions are also contemplated). For example,FIG.1Ashows an exploded view104that shows garment-integrated capacitive sensor assembly102D having four distinct garment-integrated capacitive sensor contact regions106A-106D, where each respective contact region of each respective garment-integrated capacitive sensor can be used to a value of capacitance. These garment-integrated capacitive sensors can be used to determine finely applied forces (e.g., finger rolling on a surface), which can be used for providing inputs to control an artificial-reality environment.

As will be explained in further detail in conjunction with describing subsequent figures, the one or more garment-integrated capacitive-sensor assemblies102A-102E in each respective fingertip are seamlessly integrated with knitted wearable-glove device100. This seamless nature is illustrated in exploded view104, and the exploded view104shows that the one or more garment-integrated capacitive sensor assemblies are each constructed from two knitted layers. The first-knitted-conductive-electrode layer108is constructed using an insulated-conductive fabric and the second-knitted-conductive-electrode layer110is constructed using a non-insulated-conductive fabric containing. When combined, the first-knitted-conductive electrode layer108is configured to be directly in contact with the second-knitted-conductive-electrode layer110to produce a garment-integrated capacitive sensor. While the second-knitted-conductive-electrode layer110is shown as being the external layer (i.e., on the exterior of the glove100) in the example embodiment ofFIG.1A, an opposite arrangement would also work (i.e., the insulated layer is non-external layer) in some other embodiments.

Turning now toFIG.1B, shown there is a pair of knitted wearable-glove devices100, where one glove is shown from a perspective view that depicts a palmar side of the knitted wearable-glove device100and the other glove is shown from a perspective view that depicts a dorsal side of the knitted wearable-glove device100. The first glove112illustrating the palmar side shown was discussed above in reference toFIG.1A. The second glove114illustrating the dorsal side is now described. The second glove114illustrating the dorsal side shows additional garment-integrated capacitive sensors116A-116L being disposed on one of its surfaces. As shown in this example, the capacitive sensors can be located proximate to (e.g., within 0.2-5 mm of, or directly over) on the user's joints (e.g., knuckles), such that the capacitive sensors can be used to measure bending/stretching occurring at or near the joints. As will be discussed further below, the knitted wearable-glove device100(including the garment-integrated capacitive sensors) can be produced by a multi-dimensional knitting machine (e.g., v-bed or x-bed knitting machines), allowing for the knitted wearable-glove device100that includes one or more garment-integrated capacitive sensors to be produced in a single knitting process (e.g., the wearable glove does not need to be removed from the knitting machine to be reoriented or completed). In some embodiments, additional garment-integrated capacitive sensors are located on the thumb portion of the glove as well as on the palm and wrist to provide additional input areas or sensor-detection regions to allow for further flexibility in interacting with artificial-reality environments.

In one example, the soft capacitive sensors that are integrated with the gloves ofFIGS.1A and1Bcan be used to assist with typing operations. This is shown inFIGS.1C and1D, which illustrate an example of using data provided by the one or more garment-integrated capacitive sensors to provide an input to a user interface presented in an artificial-reality environment.FIG.1Cshows a user118wearing the knitted wearable-glove device100pressing down on a surface120(e.g., a desk). The one or more garment-integrated capacitive sensor assemblies102A-102E provide data (e.g., respective capacitance measurements resulting from the user pressing down on the surface120, which measurements can be divided into the various contact regions of each of the capacitive sensors, as was noted above) indicating that a force is being applied to the fingertip122of the knitted wearable-glove device100. The measurement data can then be used to calculate force values, as is shown by curve123in plot121, which illustrates the detected force being received in response to a touch event.FIG.1Calso shows that in response to the force being applied to the fingertip122of the knitted wearable-glove device100, an input on a virtual key of a virtual keyboard124is provided to the virtual display (e.g., indicated by the letter “H”125being displayed).

FIG.1Dshows a user118who is wearing the knitted wearable-glove device100no longer pressing down on surface120(e.g., a desk). In response to no longer pressing down on the surface120, one or more garment-integrated capacitive sensor assemblies102A-102E provide data (e.g., a respective capacitance measurement) indicating that a force is not being applied to the fingertip122of the knitted wearable-glove device100. The measurement data can then be used to calculate force value, as is shown in plot126, which illustrates the detected force being received in response to a touch event ending. The detected force is now, as a result, less than the calculated force shown in plot121inFIG.1C.

FIG.1Eillustrates a lock-and-key knitting technique, which can be utilized to increase contact surface area of one or more adjacent garment-integrated capacitive sensor assemblies (e.g.,102A-102E shown inFIG.1A).FIG.1Eshows two different garment-integrated capacitive sensor assemblies (e.g., a first garment-integrated capacitive sensor128and a second garment-integrated capacitive sensor130) where first garment-integrated capacitive sensor128terminates with a first pattern132(e.g., a key) and the second garment-integrated capacitive sensor130terminates with a second pattern134(e.g., a lock) that corresponds to the first pattern (e.g., an opposite pattern than the first pattern). While described in this example as used to knit two sensor assemblies together, it should be understood that the lock-and-key knitting technique can also be used to knit two contact regions within a single sensor assembly together.

FIG.1Ealso shows the first garment-integrated capacitive sensor128and the second garment-integrated capacitive sensor130pieced together in combined garment-integrated capacitive sensor assembly136. In some embodiments, this lock-and-key structure improves knit resolution by taking advantage of the three-dimensional nature of the knits. In some embodiments, this lock-and-key structure increases a surface area of contact between the first-knitted-conductive electrode layer108and the second-knitted-conductive-electrode layer110of the one or more garment-integrated capacitive sensor assemblies102A-102E.

Attention is now directed toFIG.2, which illustrates a multi-dimensional knitting machine configured to produce multi-dimensional knitted garments in an automated fashion (e.g., with the needing for any hand knitting or other user intervention after initiating the knitting process, including allowing for having an electronic component automatically knitted as an integrated component of the multi-dimensional knitted garments), in accordance with some embodiments. The multi-dimensional knitting machine200is a garment-producing device that is computer controlled and user programmable to allow for complex knitted structures to be produced (e.g., gloves, tubular fabrics, fabrics with embedded electronic devices, complex knit patterns, special stretch characteristics, unique pattern structures, multi-thread structures, etc.,). The multi-dimensional knitting machine200includes a first-axis needle bed202, a second-axis needle bed208, and N-axis needle bed (indicating more than three needle beds are possible). Each one of these needle beds (e.g., needles204, needles210, and needles218) is configured to use multiple different types of knit patterns (e.g., jersey knits, rib knits, interlock knits, French-terry knits, fleece knits, etc.,) based on a programmed sequence providing to the multi-dimensional knitting machine200, and variations of these knits can be employed to form a single continuous garment (e.g., a combination of jersey knits and French terry knit and/or a first variation of a jersey knit and a second variation of a jersey knit). In some embodiments, the variations of these knits in a single continuous garment can be done without producing seams (e.g., a seamless wearable device can be produced). In some embodiments, the knitting machine is further configured to layer fabrics to produce multilayered wearable structures (e.g., to house one or more electronic components). In some embodiments, each layer in a multilayered wearable structure can be made from a different fabric, which in one example is produced using a conductive yarn. For example, a two-layer knitted capacitive sensor can be produced using the multi-dimensional knitting machine200, where the first layer and the second layer use different thread (e.g., a coated-conductive thread and an uncoated-conductive thread). A plurality of fabric spools (e.g., fabric spools204, fabric spools212, and fabric spools220) can be included for each one of the needle beds. Multiple types of fabric spools can be used for each needle bed allowing for even more complex woven structures (also referred to as garments) to be produced. In some embodiments, the fabric spools can also include elastic thread allowing for stretchable fabrics and/or fabrics with shape memory to be produced.

Each of the needle beds discussed above can also include one or more non-fabric insertion components (e.g., non-fabric insertion components206, non-fabric insertion components214, and non-fabric insertion components222) that are configured to be used to allow for insertion of non-fabric structures into the needle beds, such that the non-knitted structure can be knitted into the knitted structure, while the knitted structure (e.g., garment) is being produced. For example, non-fabric structures can include flexible printed circuit boards, rigid circuit boards, conductive wires, structural ribbing, sensors (e.g., neuromuscular signal sensors, light sensors, PPG sensors, etc.,), etc. In some embodiments, a stitch pattern can be adjusted by the multi-dimensional knitting machine (e.g., in accordance with a programmed sequence of knit instructions provided to the machine) to accommodate these structures, which, in some embodiments, means that these structures are knitted into the fabric, instead of being sewn on top of a knitted fabric. This allows for garments to be lighter, thinner, and more comfortable to wear (e.g., by having fewer protrusions applying uneven pressure to the wearer's skin). In some embodiments, these multi-dimensional knitting machines can also knit knitted structures along either or both of a vertical axis or a horizontal depending on desired characteristics of the knitted structure. Knitting along a horizontal axis means that the garment would be produced from a left side to a right side (e.g., a glove would be produced starting with the pinky finger, then moving to the ring finger, then middle finger, etc. (e.g., as shown in the example sequence ofFIG.3B)). Sewing on the vertical means that the garment is produced in a top-down fashion (e.g., a glove would be produced starting from the top of the tallest finger and move down to the wrist portion of the glove (e.g., as shown by228inFIG.2)). With respect to the glove examples, a reverse manufacturing process is also contemplated (e.g., knitting a thumb first when knitting on the horizontal and knitting the wrist portions when knitting on the vertical). In some embodiments, the insertion component can feed the non-knitted structure to the knitting machine or, in some other embodiments, the insertion component is fed through the knitting machine with the non-knitted structure. In the latter, the insertion component is not integrated into the garment and is discarded. In some embodiments, the insertion component is not fed at all, but is an integrated component of the multi-dimensional knitting machine that is activated based on a programming knit sequence to then allow for insertion of a non-knitting component into a knitted structure.

The multi-dimensional knitting machine200also includes knitting logic module224, which is a module that is user programmable to allow for a user (which can be a manufacturing entity producing wearable structures on mass scale) to define a knitting sequence to produce a garment using any of the above-described materials, stitch patterns, knitting techniques, etc. As stated above, the knitting logic module224allows for a seamless combination of any of the above-described techniques, thereby allowing unique complex knitted structures to be produced in a single knitting sequence (e.g., the user does not need to remove the knitted structure, then reinsert and reorient it to complete knitting the knitted structure). The multi-dimensional knitting machine200also includes insertion logic module226, which works in tandem with the knitting logic module224, to allow for insertion of non-fabric components to be seamlessly inserted into the knitted structure while the knitted structure is knitted together. The insertion logic is in communication with the knitting logic to allow for the knit to be adjusted in accordance with where the non-fabric structure is being inserted. In some embodiments, the user need only show where the non-fabric structure is to be inserted in their mock-up (e.g., at a user interface associated with the multi-dimensional knitting machine, which user interface allows for creating and editing a programmed knit sequence) and the knitting logic module224and insertion logic module226automatically work together to allow for the knitted structure to be produced.

FIG.3Aillustrates a sequence in which a knitted wearable structure (e.g., a glove) is knitted along a vertical axis, in accordance with some embodiments. Sewing on the vertical means that the garment is produced in a top-down fashion (e.g., a glove would be produced starting from the top of the tallest finger and move down to the wrist portion of the glove). For some knitted wearable structures, it is necessary to knit the knitted structure on the vertical. Sewing on specific axes can be necessary for certain three-dimensional knitted structures (e.g., pockets with certain openings).FIG.3Ashows a sequence300that shows three snapshots (302A-302C) over time of sewing along the vertical axis.

FIG.3Billustrates a sequence in which a knitted wearable structure (e.g., another glove) is knitted along a horizontal axis, in accordance with some embodiments. Sewing on the horizontal means that the garment would be produced from a left side to a right side (e.g., a glove would be produced starting with the pinky finger, then moving to the ring finger, then middle finger). For some knitted wearable structures, it is necessary to sew the knitted structure on the horizontal instead of on the vertical.FIG.3Bshows a sequence304that shows two snapshots (306A-306B) over time of sewing on the horizontal. It should be understood that certain multi-dimensional knitting machines can be programmed to allow for combinations of knitting on both the horizontal and the vertical, even for a single wearable structure, such that certain aspects of the wearable glove (e.g., a three-dimensional volumetric pocket) can be knit on the horizontal and other aspects of the wearable glove (e.g., a non-knitted structure, such as a printed circuit board) can be knit on the vertical.

FIG.4illustrates a non-knitted structure being inserted into a multi-dimensional knitting machine while knitting a knitted structure (e.g., and also doing so in an automated fashion such that no user intervention is needed to allow for integrating the non-knitted structure after the knit sequence is initiated), in accordance with some embodiments.FIG.4shows a schematic overview figure, similar to that ofFIG.2, which shows an insertion component402(same as insertion components206,214, and222described in reference toFIG.2) configured to work with a multi-dimensional knitting machine400(same as multi-dimensional knitting machine200discussed in reference toFIG.2).

FIG.4shows a sequence of how a non-knitted structure405can be inserted into the multi-dimensional knitting machine at the same time the knitted fabric (e.g., glove408) is being knitted. In the first pane406A, indicating a first point in time, the fingertips of the glove408are being produced, e.g., knitted on a vertical axis. The second pane406B, indicating a second point in time, shows the knitted structure405being knitted and also show that the non-knitted structures408A and408B (which in this example can be individual conductive traces or printed circuit boards that can be used to route data from a sensor, such as one or more of the soft capacitive sensors described earlier as one example) are being knitted into (i.e., inserted into) the knitted structure405. The second pane406B also shows that in some embodiments the thread of the knitted structure405is alternatingly knitted over or knitted under the non-knitted structure405ensuring that the non-knitted structures408A and408B is integrated into the single layer of fabric. The third pane406C, indicating a third point in time, further shows that the knitted structure405is continued to be knitted together and the non-knitted structures408A and408B are continued to be knitted into the knitted structure405. Eventually, the multi-dimensional knitting machine400along with the insertion component402will produce a complete glove408(e.g., a three-dimensional glove) with an embedded non-knitted structures408A and408B. In some embodiments, the non-knitted structures408A and408B can include cutouts to allow for thread to pass through to further secure the non-knitted structures408A and408B to the knitted fabric405.

FIG.4also illustrates how the multi-dimensional knitting machine200can adjust a knit pattern to a different knit pattern while still allowing for a non-knitted structure to be integrated into the knitted structure, in accordance with some embodiments.FIG.4illustrates in a fourth pane406D, indicating a fourth point in time, that a second knit pattern412can be switched to mid-knitting without having a seam between the two knit patterns (e.g., knit patterns can be changed and seamless knit can still be produced). Changing the knit patterns mid-knitting can be beneficial for accommodating different flex requirements of the wearable structure (e.g., locations on a glove that correspond to a joint can require a different knit pattern to accommodate more movement than locations that correspond to a phalange). In some embodiments, the first knit pattern414is a tighter (e.g. denser) knit (e.g., a higher number of individual stitches per a certain area of space) than a second-knit pattern416to accommodate additional movement (e.g., flexing of the user's joints). In some embodiments, the non-knitted structure405(e.g., a printed circuit board, an electrical wire, a bundle of electrical wires, semi rigid support, etc.) is constructed from a material that is different from the fabric structure. For example, the non-knitted structure can be a printed circuit board, an electrical wire, a bundle of wires, a semi-rigid support, etc.

FIGS.5A and5Billustrate a knitted structure with a non-knitted structure where the non-knitted structure has a first knit portion surrounding it and a second knit portion surrounding the first knit portion, in accordance with some embodiments.FIG.5Aillustrates a structure500that includes a non-knitted structure502and knitted structure501. The non-knitted structure502, in the example depicted inFIGS.5A-5B, does not stretch like the knitted structure501, so it is necessary to devise a technique that allows the knitted structure to stretch while not damaging the non-knitted structure502.FIG.5Ashows a knitted structure501that uncoils a non-knitted structure502while the knitted structure501is stretched, effectively allowing the non-knitted structure502to not interfere with (i.e., match) the stretch of the knitted structure501.FIG.5Ashows a first knitted portion506that has a first knit pattern and the second knitted portion508that has a second knit pattern (different from the first knit pattern) that accommodates the non-knitted structure502. The second knitted portion508, when combined with the non-knitted structure502, is configured to stretch in substantially the same manner as the first knitted portion506. For example, a looser knit pattern may be used in the second knitted portion508to accommodate the reduced stretch capabilities resulting from the non-knitted structure502. To allow the non-knitted structure502to not receive undue stress when stretched, the non-knitted structure502can be oversized for a given area and placed in a meandering pattern. The meandering pattern allows the non-knitted structure502to move when the fabric is being stretched, without putting excess stress/strain on the non-knitted structure (e.g., making the max stretch length equal to the non-knitted structures length when linear). Undue stress and/or strain on knitted structures can damage components or interfere with accurate measurements when the non-knitted structures are used for sensing purposes.

FIG.5Billustrates the structure500, described in reference toFIG.5A, being in a stretched state (e.g., as indicated by opposing arrows510A and510B), which shows the knitted structure501being stretched and the non-knitted structure502in its extended state.FIG.5Aalso shows that first knitted portion506and the second knitted portion508are stretched in the horizontal direction. While stretch is shown in one direction, in some embodiments, the fabric can also be configured to have two-way stretch.

FIGS.6A-6Billustrate a first kind of stitch pattern (e.g., a jersey stitch pattern) that is utilized to allow for accommodating a conductive trace, in accordance with some embodiments. In some embodiments, the conductive trace can be knitted into a yarn that mimics a yarn of the adjacent yarn in the fabric. In some embodiments, this yarn can be any of the conductive yarns described in Appendix A, including the yarns shown and described with reference toFIGS.3-7in Appendix A. This alternative method provides another way of providing a seamless fabric structure that includes one or more electrical components.FIG.6Aillustrates a sewing technique600that is used to produce the jersey stitched fabric.FIG.6Aalso illustrates the conductive yarn602as having a different appearance than the surrounding yarns604, i.e., to distinguish it from the surrounding yarns. However, the conductive yarn, in some embodiments, can have the same appearance. The knitting needles605shown inFIG.6A(and also pictured inFIG.6C) are the knitting needles that correspond with the needles204,210, and218described in reference toFIG.2.FIG.6Bshows the resulting fabric606constructed using a single stitch jersey stitch that includes the conductive yarn602and the surrounding yarns604.

FIGS.6C-6Dillustrate a second kind of stitch pattern (e.g., a jersey stitch pattern different than that depicted and described with reference toFIGS.6A-6B) that is utilized to allow for accommodating a conductive trace, in accordance with some embodiments. In some embodiments, the conductive trace can be knitted into a yarn that mimics a yarn of the adjacent yarn in the fabric. This alternative method provides another way of providing a seamless fabric structure that includes one or more electrical components.FIG.6Cillustrates a sewing technique608that is used to produce the jersey stitched fabric.FIG.6Calso illustrates the conductive yarn610as having a different appearance than the surrounding yarns612, i.e., to distinguish it from the surrounding yarns. However, the conductive yarn, in some embodiments, can have the same appearance.FIG.6Calso shows that some of the needles605are knitting the conductive yarn610while others are knitting the surrounding yarns612.FIG.6Dshows the resulting fabric614constructed using a modified jersey stitch that includes the conductive yarn602and the surrounding yarns604. The modified jersey stitch can allow for additional stretch (i.e., having a different stitch around the conductive yarn can improve overall stretchability of the fabric614).

FIG.6Eillustrate another example of a stitch pattern that adjusts gauge of the stitch to adjust a stretching characteristic of the resulting fabric, in accordance with some embodiments.FIG.6Eillustrates three different stitch sizes (e.g., small stitch gauge616, medium stitch gauge618, large stitch gauge620, etc.,). For example, gauges up to 18 gauge or higher. In some embodiments, different stitch gauges may be used in the same garment depending on the requirements of the area being stitched (e.g., high movement areas (e.g., joints) may require a large gauge to allow for more stretch than low mobility areas).

FIG.6Fshows an example of a fabric that includes a larger gauge knit stitch622(e.g., a larger-gauge knit jersey stitch) that allows for accommodating additional stretching characteristics, in accordance with some embodiments. This larger knit jersey stitch is apparent when compared to the jersey stich shown inFIG.6B. While a couple stitch gauges have been illustrated, any gauge can be used based on the stretch requirements of the garment. While primarily jersey stitches have been shown, other stiches, mentioned above with respect toFIG.2can be used. In some embodiments, the knit fabrics of Appendix A can be formed using this larger gauge knit jersey stitch, such as those fabrics described in reference toFIGS.3-7of Appendix A.

FIG.6Gillustrates that a conductive yarn624can be stitched in a vertical direction (e.g., along a wales direction, as opposed to a course direction), as opposed to a horizontal direction used for stitching of the conductive yarn described in reference to the examples ofFIGS.6A-6F. In some embodiments, there may be both vertical and horizontal stiches depending on the requirements of the garment. In some embodiments, the conductive yarn is coated, such that conductive yarns may be in contact with each other without interfering with their respective signals.FIG.6Hillustrates that the conductive yarn626(shaded) can be knitted yet another manner that is not jersey stitch, in accordance with some embodiments.

FIGS.7A-7Gillustrate a sequence for producing a portion of an actuator that is configured to be placed at a fingertip, in accordance with some embodiments.FIGS.7A-7Cillustrate the progression of a knitted structure being produced over time. The knitted structure700being produced consists of two different fabric components. The first fabric component702has a first knit pattern and is the desired finished fabric product in the depicted example. In some embodiments, the first fabric component702is also stitched in a manner such that a volumetric pocket is produced, i.e., to better form around a fingertip of a user when completed. The second fabric component704can be a temporary piece that is configured to be removed at a later point in production. The second fabric component704can be primarily used as a guide during an over molding step, which will be described in further detail later.

FIG.7Bbetter illustrates that the second fabric component704has its stitch pattern altered at certain locations to add in guide holes706A-706D, which are used for aligning the knitted structure in an over molding machine to consistently place the over-molded structure in the correct position.FIG.7Cfurther illustrates the knitting process continuing with more guide holes being added as the knitted structure700continues to be produced.FIG.7Calso illustrates that one or more stress relief holes707can also be produced by the multi-dimensional knitting machine. In some embodiments, these one or more stress relief holes707can be used to route cable (e.g., electronic, fluid, or pneumatic cables), or be used to allow for the fabric to be bent (e.g., bent around a tip of a finger).

FIG.7Dillustrates the knitted structure700being inserted into an over molding machine708to have one or more haptic feedback generator components710integrated into the first fabric component702. As discussed, the second fabric component704includes guide holes706A-706L that correspond to dowels712A-712L. The dowels712A-712L are inserted into the guide holes706A-706L of the second fabric component704to ensure that the over molding machine708correctly places the over-molded structure onto the first fabric component702.

FIG.7Eshows the over molding machine708compressing down on the knitted structure700(not visible inFIG.7Eas it has been compressed by the machine708) to inject the over-molded structure (not visible inFIG.7Eas it has been compressed by the machine708). While compressing down on the knitted structure700, an injectable material with bending properties (e.g., silicone, rubber, etc.,) can be flowed onto/into the fabric in the shape of the mold provided by the over molding machine708to produce the over-molded structure (obscured). In some embodiments, an additional component is added to the over molding machine and the over molding machine then secures the additional component to the knitted structure700via the molded structure.

FIG.7Fillustrates the post-over-molded structure that now includes over-molded structure714embedded into the first fabric component702of the knitted structure704to produce a completed haptic fingertip structure716. This over-molded structure714can be configured to include a matrix of haptic-feedback generators (e.g., as illustrated by the bubble array717, where individual bubbles can each be used to provide haptic feedback and/or to sense inputs), where each haptic feedback generator can be individually controlled (e.g., by inflating or deflating) to provide a haptic sensation to a user wearing the completed haptic fingertip structure716. In some embodiments, the over-molded structure714includes one or more sensors (e.g., a neuromuscular signal sensor that is secured during the over molding process). In some embodiments, the one or more sensors are configured to detect both neuromuscular signals and non-neuromuscular signals.FIG.7Falso shows two strings718A and718B, which are configured to be the only strings, in this example embodiment, attaching the first fabric component702with the second fabric component704. In some embodiments, there is only a single string attaching the first fabric portion component702with the second fabric component704.

FIG.7Gshows that the two strings718A and718B being pulled from the knitted structure700and as a result the first fabric component702and the second fabric component704become detached from one another. In some embodiments, a single string can be configured to detach the first fabric component702from the second fabric component704. In some embodiments, the second fabric component is one continuous piece instead of two separate pieces. As stated earlier, the second fabric component704is a temporary piece that is configured to be removed at a later point in production and is used only during the manufacturing process.FIG.7Galso shows the completed process of producing a fabric with an integrated over-molded structure720. In some embodiments, strings718A and718B are loose strings from the704second fabric component, which allow for unraveling (e.g., by hand or machine) of the second fabric component704in order to detach the second fabric component704from the first fabric component702.

FIGS.8A-8Billustrate a fabric structure (e.g., a glove800) that includes one or more portions that are made from a conductive deformable fabric (e.g., conductive deformable fabric portion802) and favorable strain characteristics accommodated by this fabric structure, in accordance with some embodiments. In some embodiments, the conductive deformable fabric has a different amount of stretch along certain axes (e.g., more restrictive) compared to the surrounding material, but a stretch is still desired. In order to integrate the conductive deformable fabric, certain folding techniques can be used to achieve this goal, such as origami derived folding techniques (e.g., the fabric has alternating folds along at least one axis to reduce its footprint (e.g., a first footprint) when in an unstretched state and when in a stretched state the alternating folds substantially unfold to increase its footprint (e.g., to a second footprint, larger along at least one axis than the first footprint) of the fabric). In some embodiments, the fabric structure includes elastic, allowing the fabric to be in a default unstretched state.

In some embodiments, the conductive deformable fabric portion802can be configured to be a strain sensor (i.e., based on the unfolding of the fabric the resistance of the fabric changes, which can be used to determine the strain occurring). In some embodiments, the strain information can be used to determine pose of a hand (e.g., the strain can be used to determine whether the fingers are in a curled/first state (e.g., higher strain, fingers more tightly curled)). In some embodiments, the conductive deformable fabric can also be configured to couple with a neuromuscular signal sensor, and the conductive deformable fabric can be configured to power the neuromuscular signal sensor and/or transmit signal data from the neuromuscular signal sensor.

FIG.8Aalso shows a user801curling their hand into a first and as a result portions of the glove800are extended into a stretched state. The plot804shows a prophetic illustration of a curve806indicating the measured/calculated strain (shown on y-axis808) occurring at the conductive deformable fabric portion802over time (shown on x-axis810). The curve806shows that the strain increases as the hand tightens further (i.e., the conductive deformable fabric portion802is put further into its stretched state). In some embodiments, multiple discrete strain sensors can be placed in different areas as opposed to a continuous strip, as pictured (e.g., an individual strain sensor placed on each joint, or other flexible part of the hand) to take multiple strain measurements and provide an even better picture of the hands pose. In some embodiments, multiple strain sensors may be placed at a single location (e.g., a joint) to provide an even further detailed (e.g., higher resolution) measurement. In some embodiments, the information provided by the one or more strain sensors can be used to provide an input into an artificial reality environment displayed at an artificial-reality headset803.

FIG.8Bshows a user now uncurling their hand and as a result the glove800is returned to its unstretched state. The plot804shows the prophetic curve806now indicating the measured/calculated strain occurring at the conductive deformable fabric portion802over time. The curve shows that the strain decreases as the hand uncurls further (i.e., the conductive deformable fabric portion802is put further into its unstretched state).

FIGS.9A-9Cillustrate a fabric structure900that includes one or more portions that are made from a conductive deformable fabric902and the fabric structure900is configured to have two-way stretch with favorable strain characteristics shown by the plots in each ofFIGS.9A-9C, in accordance with some embodiments. As discussed in reference toFIG.8A-8B, the fabric structure900is made stretchable by using a series of folds and the unstretched state is a substantially folded state and the stretched state is a substantially unfolded state. As will be discussed, the fabric structure900can have a folding pattern that allows it to be unfolded in both the x-direction and y-direction allowing for two-way stretch.

FIG.9Ashows the fabric structure900in a default unstretched state. The plot904shown inFIG.9Aindicates by dashed x-axis curve906and solid y-axis curve908that at time t1there is no measured/calculated strain occurring in both the x and y axes of the fabric structure, respectively.

FIG.9Bshows the fabric structure900in an extended state along the y-axis, and the plot904shown inFIG.9Bindicates by solid y-axis curve908that at time t2there is a measured/calculated strain along the y-axis.FIG.9Balso shows that dashed x-axis curve906at time t2indicates that there is no measured/calculated strain occurring along the x-axis.

FIG.9Cshows that the fabric structure900is in an extended state along both the x-axis and y-axis, and the plot904shown inFIG.9Cindicates by dashed x-axis curve906and solid y-axis curve908that at time t3there is both a measured/calculated strain occurring in both the x and y axes of the fabric structure, respectively.

FIG.10Aillustrates two views of a knitted fabric that includes a volumetric knit that can be configured to house one or more non-knitted structures, in accordance with some embodiments. First view1000shows a top-down view of a knitted structure1002that includes a volumetric portion1004. The volumetric portion1004acts as a pocket allowing for a non-knitted structure (not pictured) to be placed within the cavity of the volumetric portion. In some embodiments, the non-knitted structure is inserted via an insertion component. In some embodiments, the non-knitted structure is a neuromuscular signal sensor (e.g., an electromyography sensor).

FIG.10Aalso pictures a second view1006that shows a side view of the knitted structure1002that includes the volumetric portion1004. In some embodiments, the knitted structure1002is produced on a multi-dimensional knitting machine that is configured to adjust its knit pattern while producing the knitted structure to produce the volumetric portion1004. In some embodiments, the volumetric portion has no seam or boundary with the adjacent portion, as it produced by only changing the knit pattern (e.g., a denser knit pattern surrounded by portions that have a looser knit pattern can produce a volumetric pocket).

FIG.10Bshows an embodiment where multiple volumetric portions are placed on a single knitted structure1008, in accordance with some embodiments. Multiple volumetric portions1010A-1010C can allow for multiple non-knitted structures to be placed in close proximity to each other. In some embodiments, the volumetric portions are placed in a grid array spanning both x and y directions. In some embodiments, the volumetric portions are offset from each other along one or more axes.

The above descriptions complement the numerous manufacturing procedures and yarn types described in Appendix A, such that the various yarns (e.g., the different yarn materials that can be used described in reference toFIGS.3-7of Appendix A) and manufacturing procedures (e.g., laser cutting, die cutting, and forming electrical connections discussed generally in reference toFIGS.8-55in Appendix A) can be used in conjunction with the textile structures and manufacturing processes discussed elsewhere herein, and Appendix A is appended to this specification.

FIG.11illustrates method flow chart1100for detecting force received at a garment, in accordance with some embodiments.

(A1) In accordance with some embodiments, a method of detecting force received at a garment (1102), comprises, receiving (1104) a force at a sensor integrated into a garment, wherein the capacitive sensor includes: a first knitted conductive electrode layer that is constructed using an insulated conductive fabric, where the first knitted conductive electrode layer has a first surface, and a second knitted conductive electrode layer that is construed using a non-insulated conductive fabric containing a second surface, where the second surface is configured to be directly in contact with the first surface (e.g., knitted onto the same layer as the first layer, where the first layer is a structural component of a wearable device (e.g., glove)) to produce the sensor. The method also includes that in response to receiving the force at the sensor, transmitting (1106) a value corresponding to the received force to a processor. The method then includes determining (1108), via the processor, a calculated force value. More detail on the capacitive sensor of A1 is provided below in reference to B1 through B17. Appendix A provides for further details on example materials used for producing textile-based electrodes, such that any of the example materials shown and described in Appendix A could be used in conjunction with the other textile structures described herein and/or in conjunction with the manufacturing processes and techniques described herein as additions to or alternatives to the manufacturing processes and techniques described herein. For example, the conductive yarns (e.g., silvertech+150-22 Tex or Statex Shieldex Yarn 235/36 1-Ply) described in reference toFIGS.3-7of Appendix A.

(B1) In accordance with some embodiments, a garment-integrated capacitive sensor, comprises a first knitted conductive electrode layer that is constructed using an insulated conductive fabric (e.g., insulated conductive fabric can be made with a compressible/stretchable core (e.g., elastane, thermoplastic polyurethane (TPU)) that enables deformation at the yarn level, which enhances the capacitive sensors performance. In some embodiments, high surface area insulated conductors (e.g., enamel coated copper foil, etc.) wrapped around the core further improves sensor performance. In some embodiments, silver-copper alloy wires/foils provide balanced performance when electrical conductivity, cost, and fatigue resistance are considered compared to pure copper, tin copper alloy, and silver copper alloy). The first knitted conductive electrode layer has a first surface. The garment-integrated capacitive sensor also comprises a second knitted conductive electrode layer that is constructed using a non-insulated conductive fabric containing a second surface, the second surface is configured to be directly in contact with the first surface to produce a garment-integrated capacitive sensor. In some embodiments, the garment-integrated capacitive sensor is configured to be in communication with a processor, and is configured to receive a sensed value from the garment-integrated capacitive sensor.

For example,FIGS.1A-1Dillustrate examples of garment-integrated capacitive sensors integrated into a wearable device and their uses, in accordance with some embodiments.

In some embodiments, the second knitted conductive electrode layer is constructed using material such as: silver, platinum, gold, etc. In some embodiments, a coating/plating is applied at each fiber level (e.g., each fiber of the knitted conductive electrode is coated/plated). In some embodiments, solderable yarns enable easier electrical interconnections. In some embodiments, the second knitted conductive electrode layer is constructed using conductive yarns made from silver-plated nylon. In some embodiments, first knitted conductive electrode layer and second knitted conductive electrode layer are made from yarns/wires with a TPU core, and the TPU core allows for tunable compressibility. In some embodiments, electrical interconnects are made using ultrasonic bonding. In some embodiments, conductive or insulated conductive wire/foil is wrapped around it using a yarn cover/twist machine.

A garment-integrated capacitive sensor without separate dielectric has the capability to conform around the human body easier (e.g., curved portions such as a fingertip). In some embodiments, the textile sensors with custom shapes are knitted seamlessly as part of the substrate (e.g., glove fingertips, wristband), which is built in a single manufacturing step (e.g., a single knitting sequence). Some drawbacks of using dielectric film, such as 3-layer sensor geometry, in the sensor construction means that every time the sensor needs to be knitted, the machine has to be stopped and the dielectric film needs to be inserted manually in between electrodes. Another drawback of the three-layer design is that since the space for the dielectric film to be inserted is only a couple of millimeters, the dielectric film may not be inserted properly. When the dielectric film is not be inserted properly, the sensor can short. In addition, it is difficult to diagnose improper construction of the three-layer design until the whole glove/sensor swatch is knitted. Moreover, this step requires preparation of custom sized dielectric films to accommodate different shape/size sensors. In addition, the three-layer sensor configuration is more time consuming to produce and is more difficult to automate manufacturing.

(B2) In some embodiments of B1, the sensed value when processed by the processor can infer a force received at the garment-integrated capacitive sensor. For example,FIGS.1C and1Dshow in plot121and plot126, respectively, show a determined force being received at the glove100.

(B3) In some embodiments of any of B1-B2, the sensed value when processed by the processor can determine if the garment-integrated capacitive sensor is in contact with a surface. For example,FIG.1Cshows that in response to the glove100being in contact with a surface120at a location that corresponds to a virtual key of a virtual keyboard124, a “H” letter125is displayed on a display (e.g., a real display or a display shown in an artificial reality).

(B4) In some embodiments of any of B1-B3, the processor is in further communication with an artificial-reality headset displaying an artificial reality, and the sensed value from the garment-integrated capacitive sensor is used to alter a visual aspect of the artificial reality.FIG.1Cshows an example of the glove providing an input (e.g., “H” letter125is displayed on a display) to a display via a virtual keyboard.

(B5) In some embodiments of B1-B4, the garment-integrated capacitive sensor is seamlessly knitted into a fabric that is not a capacitive sensor. For example,FIGS.1A and1Bshow that a knitted wearable-glove device100that includes one or more garment-integrated capacitive sensors, where the garment-integrated capacitive sensors are seamlessly integrated (e.g., at least on surface of the glove does not have a raised bead/stitch for tying the thread of the glove with the one or more garment-integrated capacitive sensors).

(B6) In some embodiments of B1-B5, the garment-integrated capacitive sensor is integrated into a wearable device (e.g., glove100shown inFIGS.1A-1D), wherein the wearable includes a plurality of garment-integrated capacitive sensors. In some embodiments, the plurality of garment-integrated capacitive sensors can be split into quadrants, wherein the quadrants are configured to wrap around a fingertip a three-dimensional manner. In some embodiments, the plurality of garment-integrated capacitive sensors are knitted together continuously.

(B7) In some embodiments of B1-B6, each of the plurality of garment-integrated capacitive sensors can detect a pressure covering an area between 0.5-15 cm2.

(B8) In some embodiments of B1-B7, the second surface is configured to be directly in contact with the first surface without a separate dielectric sheet. For example,FIG.1Ashows a two-layer capacitive sensor that has first-knitted-conductive-electrode layer108that is constructed using an insulated-conductive fabric and the second-knitted-conductive-electrode layer110that is constructed using a non-insulated-conductive fabric containing.

(B9) In some embodiments of B1-B8, the garment-integrated capacitive sensor is integrated into a wearable glove (e.g., glove100inFIGS.1A-1D).

(B10) In some embodiments of B9, an additional garment integrated capacitive senor is integrated into the wearable glove (e.g.,FIGS.1A and1Bshow a plurality of garment-integrated capacitive-sensor assemblies (e.g.,102A-102E inFIG.1Aand garment-integrated capacitive sensors116A-116L inFIG.1B).

(B11) In some embodiments of B10, the garment-integrated capacitive sensor and the additional garment integrated capacitive senor are located in separate fingertips of the wearable glove (e.g., garment-integrated capacitive-sensor assemblies102A-102E). In some embodiments, sensors are located at each fingertip of the glove. In some embodiments, the sensors are located on the palmar side of the hand or on the dorsal side of the hand.

(B12) In some embodiments of B1-B9, the garment-integrated capacitive sensor is knitted together with a non-sensor portion of a garment using a v-bed knitting machine (e.g.,FIG.2shows that a glove is being produced using a multi-dimensional knitting machine).

(B13) In some embodiments of B12, multiple garment-integrated capacitive sensors are knitted together with a non-sensor portion of a garment using a v-bed knitting machine (e.g.,FIG.2shows that a glove is being produced using a multi-dimensional knitting machine).

(B14) In some embodiments of B13, the multiple garment-integrated capacitive sensors are knitted together using a lock and key knit pattern (e.g., a lock and key knit pattern increases the active surface area of the multiple garment-integrated capacitive sensors, thereby improving performance). In some embodiments, the lock and key knit pattern can be applied to improve energy storage of parallel electrodes, knitted components used for energy harvesting, etc.,FIG.1Eillustrates a lock and key structure for connecting multiple integrated capacitive sensors.

(B15) In some embodiments of B1-B9, the insulated conductive fabric is constructed of a conductor that is coated with an insulating material. For example, a first-knitted-conductive-electrode layer108that is constructed using an insulated-conductive fabric is discussed in reference toFIG.1A.

(B16) In some embodiments of B15, the insulating material does not alter the pliability of the conductive fabric.

(B17) In some embodiments of B1-B9, the insulated conductive fabric is constructed of a conductor with an insulated shroud surrounding the conductive fabric.

FIG.12illustrates method flow chart1200for manufacturing a knitted fabric that includes a non-knitted structure, in accordance with some embodiments.

(C1) In accordance with some embodiments, a method (1200) of manufacturing a knitted fabric that includes a non-knitted structure comprises, while knitting a fabric structure in accordance with a programmed knit sequence for a V-bed knitting machine (e.g., or any other suitable multi-dimensional knitting machine) (1200): providing (1204) a non-knitted structure to the V-bed knitting machine at a point in time when the fabric structure has a first knit portion, wherein the first knit portion is formed based on a first type of knit pattern, and after the providing of the non-knitted structure, following (1208) the programmed knit sequence to automatically adjust the V-bed knitting machine to use a second type of knit pattern, distinct from a first type of knit pattern, to accommodate the non-knitted structure within a second knit portion that is adjacent to the first knit portion within the fabric structure. For example,FIG.2illustrates a multi-dimensional knitting machine200that includes multiple non-fabric insertion components for inserting non-fabric components into a knitted fabric.FIG.4also illustrates an example of how a non-knitted structures408A and408B can be knitted into (i.e., inserted into) a knitted structure405(e.g., a glove).

(C2) In some embodiments of C1, the non-knitted structure is provided to the V-bed knitting machine via an insertion device that is distinct from the V-bed knitting machine (e.g.,FIG.4illustrates an insertion component402that corresponds to a multi-dimensional knitting machine400).

(C3) In some embodiments of any of C1-C2, the insertion device is passed through the V-bed knitting machine.

(C4) In some embodiments of any of C1-C3, the insertion device is attached to the V-bed knitting machine and feeds the non-knitted structure into the v-bed knitting machine in accordance with the programmed knit sequence (e.g.,FIG.4illustrates that an insertion component402cam be mounted above one of the knitting beds of the multi-dimensional knitting machine402).

(C5) In some embodiments of any of C1-C4, the first type of knit pattern has a higher knit density than the second type of knit pattern.

(C6) In some embodiments of any of C1-C5, the first type of knit pattern uses a type of knit pattern than stretches more (or less) than the second type of knit pattern (e.g.,FIG.4shows in fourth pane406D that a first-knit pattern414has a tighter (e.g. denser) knit than second-knit pattern416to accommodate additional movement).

(C7) In some embodiments of any of C1-C6, the non-knitted structure is a flexible circuit board (e.g.,FIG.4shows that a non-knitted structure405(e.g., a printed circuit board) is being inserted into the knitted structure).

(C8) In some embodiments of any of C1-C7, the non-knitted structure is an electrical wire or bundle of electrical wires (e.g.,FIG.4shows that a non-knitted structure405(e.g., an electrical wire or a bundle of electrical wires) is being inserted into the knitted structure).

(C9) In some embodiments of any of C1-C8, the non-knitted structure is a semi-rigid support for providing rigidity to the fabric structure (e.g.,FIG.4shows that a non-knitted structure405(e.g., a semi-rigid support) is being inserted into the knitted structure).

(C10) In some embodiments of any of C1-C9, the first knit portion and the non-knitted structure within a second knit portion have substantially the same stretchability (e.g., one-way or two-way stretch). For example,FIGS.5A-5Billustrate that a first knitted portion that does not include a non-knitted structure can stretch at the same rate as a second knitted portion that does include a non-knitted structure.

(C11) In some embodiments of any of C1-C10, including, after providing a non-knitted structure to the V-bed knitting machine at a point in time when the fabric structure has a first knit portion, formed based on a first type of knit pattern, and before following the programmed knit sequence to automatically adjust the V-bed knitting machine to use a second type of knit pattern. In some embodiments, the method also including, following the programed knit sequence to automatically create a transition area where the fabric has a second type of knit pattern, wherein the second type of knit pattern allows for more movement of the non-knitted structure. For example,FIG.4shows that the knit pattern changes to accommodate the the non-knitted structures408A and408B.

(C12) In some embodiments of any of C1-C11, the non-knitted structure is inserted such that it follows a meandering pattern along an axis, wherein the meandering pattern allows the non-knitted structure to stretch along the axis with knitted portions of the fabric structure. For example,FIG.5A-5Billustrate a meandering pattern that allows a non-knitted structure502to move when the fabric is being stretched, without putting excess stress/strain on the non-knitted structure.

(C13) In some embodiments of any of C1-C12, the second type of knit pattern can be a volumetric knit to allow for a non-knitted structure to be placed in a volume of the volumetric knit.FIGS.10A-10Billustrate that a volumetric portion1004and1010A-1010C can be produced to house one or more non-knitted structures.

(C14) In some embodiments of any of C1-C13, the programmed knit sequence for a V-bed knitting machine is configured to accommodate multiple non-knitted structures while knitting the fabric structure (e.g.,FIG.4shows non-knitted structures408A and408B being knitted into (i.e., inserted into) the knitted structure405).

(C15) In some embodiments of C14, one of the multiple non-knitted structures is a different material than the non-knitted structure (e.g., as discussed in reference toFIG.4, the non-knitted structure405can be a printed circuit board, an electrical wire, a bundle of electrical wires, semi rigid support, etc., which are different materials).

(C16) In some embodiments of C14, one of the multiple non-knitted structures is a different shape than the non-knitted structure (e.g.,FIG.4shows that the non-knitted (e.g., conductive wires, flexible printed circuit board, etc.,) structures are a different shape than the knitted structure (e.g., a glove made with thread).

(C17) In accordance with some embodiments, a knitted fabric device that includes a non-knitted structure is configured in accordance with any of C1-C16.

(D1) In accordance with some embodiments, a method of manufacturing a knitting machine, comprises, providing a V-bed knitting machine and attaching an insertion mechanism to the V-bed knitting machine. The method also includes, interconnecting the V-bed knitting machine and the insertion mechanism to a processor, wherein the processor is configured to cause a performance of a method. The method includes, while knitting a fabric structure in accordance with a programmed knit sequence for the V-bed knitting machine: providing, a non-knitted structure, via the insertion mechanism, to the V-bed knitting machine at a point in time when the fabric structure has a first knit portion, formed based on a first type of knit pattern, and after the providing of the non-knitted structure, following the programmed knit sequence to automatically adjust the V-bed knitting machine to use a second type of knit pattern, distinct from a first type of knit pattern, to accommodate the non-knitted structure within a second knit portion that is adjacent to the first knit portion within the fabric structure.

FIG.13shows a method1300for chart for knitting a dual density fabric that includes an over-molded structure, in accordance with some embodiments.

(E1) In accordance with some embodiments, a method (1300) of knitting a dual density fabric (1302), the method comprises, while knitting a fabric structure with a programmed knit sequence for a V-bed knitting machine (1304): knitting (1306) a first portion of the fabric structure with a first fabric density to include a three-dimensional pocket (e.g., the discussion in reference toFIGS.7A-7Gdescribes that the first fabric component702is stitched in a manner such that a volumetric pocket is produced), and automatically (1308) adjusting the V-bed knitting machine based on the programmed knit sequence to knit a second portion of the fabric structure with a second fabric density, distinct from the first fabric density, that is adjacent to the first portion within the fabric structure (e.g.,FIGS.7A-7Gshows a second fabric component704, which is a temporary piece). In some embodiments, the second portion is knitted first. For example, knitting the second portion of the fabric structure with the second fabric density, and automatically adjusting the V-bed knitting machine based on the programmed knit sequence to knit the first portion of the fabric structure to include a three-dimensional pocket with the first fabric density, distinct from the second fabric density, that is adjacent to the first portion within the fabric structure. The method also includes over molding (1310) a polymer over-molded structure into the three-dimensional pocket (e.g.,FIG.7F-7Gshows an over-molded structure714that can be configured to include a matrix of haptic feedback generators (e.g., as illustrated by the bubble array717)), where the second portion of the fabric structure is temporarily secured to device configured to attach the over-molded structure into the three-dimension pocket. The method also includes, removing (1312) the second portion of the fabric structure (e.g.,FIG.7Gshows that two strings718A and718B being pulled from the knitted structure700and as a result the first fabric portion component702and the second fabric component704become detached from one another).

(E2) In some embodiments of E1, the three-dimensional pocket is configured to house one or more sensors. For example, theFIG.7Fincludes an over-molded structure714that can include one or more sensors (e.g., embedded sensors).

(E3) In some embodiments of E2, the one or more sensors are neuromuscular sensors, and the neuromuscular sensors are configured to detect one or more neuromuscular signals of a user. For example, theFIG.7Fincludes an over-molded structure714that can include one or more sensors where the sensors are neuromuscular signal sensors.

(E4) In some embodiments of E2, the one or more sensors are non-neuromuscular sensors, and the non-neuromuscular sensors are configured to detect one or more non-neuromuscular signals associated with a user. For example, theFIG.7Fincludes an over-molded structure714that can include one or more sensors where the sensors are not neuromuscular signal sensors (e.g., temperature sensors, inertial measurement sensors, etc.,).

(E5) In some embodiments of any of E1-E2, the polymer over-molded structure is a component of a haptic feedback generation system. For example, theFIG.7Fincludes an over-molded structure714that can include one or more haptic feedback generators (e.g., as illustrated by the bubble array717)).

(E6) In some embodiments of E5, the haptic feedback generation system is a pressure activated system (e.g., a pneumatic or hydraulic system).

(E7) In some embodiments of E5, the haptic feedback generation system is an electrically activated system (e.g., a Dielectric Elastomer Actuator (DEA)).

(E8) In some embodiments of E5, haptic feedback generation system includes a matrix of haptic feedback generators (e.g., expandable bubbles for applying pressure to skin of a user). For example,FIG.7Fshows a bubble array717.

(E9) In some embodiments of any of E1-E2, the fabric density is determined by a combination of material weight and stitch. For example,FIGS.6A-6Hillustrate multiple types of stitches with different gauges.

(E10) In some embodiments of any of E1-E2, including, before over molding the polymer over-molded structure into the three-dimensional pocket, placing (e.g., automatically) the fabric structure in an injection molding machine (e.g.,FIG.7Dillustrates the knitted structure700being inserted into an over molding machine708).

(E11) In some embodiments of E10, the placing of the fabric structure in the injection molding machine is done based on knitted position guides (e.g. holes in the fabric) integrated into the second portion of the fabric structure. For example,FIG.7Dshows a second fabric component704that includes guide holes706A-706L that correspond to dowels712A-712L.

(E12) In some embodiments of E11, the guides are holes (or markers (e.g., a different colored thread) or a fabric bump) for securing the fabric structure in a specific location within the injection molding machine. In some embodiments, the holes are automatically knitted into the second fabric structure. For exampleFIG.7Dshows dowels712A-712L being inserted into the guide holes706A-706L of the second fabric component704to ensure that the over molding machine708correctly places the over-molded structure onto the first fabric component702.

(E13) In some embodiments of any of E1-E2, removing the second portion of the fabric structure does not compromise the first portion of the fabric structure.

(E14) In some embodiments of E13, removing the second portion of the fabric structure is done by removing a removable attachment threading (e.g.,FIG.7Falso shows two strings718A and718B, which are configured to be the only strings attaching the first fabric component702with the second fabric component704). In some embodiments of E13, removing the second portion of the fabric structure is done by unraveling the second portion of the fabric structure when pulling the string.

(E15) In some embodiments of E14, the removable attachment threading is a single thread. For example, in reference to the discussion ofFIG.7G, an alternate embodiment can include a single string that can be configured to detach the first fabric component702from the second fabric component.

(E16) In some embodiments of any of E1-E2, the first portion of the fabric structure includes a third density different than the first density. For example, the portion of the first fabric component that has a pocket (e.g., a volumetric pocket) can be done by changing the density of the fabric, similar to the volumetric pockets described in reference toFIGS.10A-10B.

(E17) In some embodiments of any of E1-E2, the first portion of the fabric structure includes one or more stress relief holes (or cuts) for wrapping the second fabric structure around a finger of a user (e.g., one or more stress relief holes707are described in reference toFIG.7C).

(E18) In some embodiments of any of E1-E2, the first portion of the fabric is configured to wick moisture away from the polymer over-molded structure. In some embodiments, reducing moisture improves performance of the haptic feedback generator.

(E19) In accordance with some embodiments, a knitted dual density fabric structure that includes an over-molded structure is configured in accordance with any of E1-E18.

Another embodiment concerning conductive deformable fabric will now be discussed below.

(F1) In accordance with some embodiments, a wearable device, comprises a conductive deformable fabric (e.g.,FIGS.8A-8Billustrate a fabric structure (e.g., a glove800) that includes one or more portions that are made from a conductive deformable fabric (e.g., conductive deformable fabric portion802), and the conductive deformable fabric comprises a conductive trace that has a non-extendable fixed length along a first axis. The conductive trace is knitted into a fabric structure to produce a conductive deformable material. The fabric structure includes a stitch pattern that facilitates the conductive trace to unfold and fold in a oscillating fashion to allow the conductive trace to expand and contract, respectively, along the first axis without exceeding the fixed length of the conductive trace, and the conductive deformable material is positioned within the wearable device such that when the wearable device is worn, the stitch pattern is over a joint of the user to allow the stitch pattern to expand or contract along with the movement of the joint. While a joint is used as a primary example of a portion of a body that can bend and cause stretching of the stitch pattern, a skilled artisan would understand that the same principles can be applied to any portion of the body that bends, expands, contracts, twists, etc., For example,FIGS.9A-9Cillustrate a fabric structure900that includes one or more portions that are made from a conductive deformable fabric902and the fabric structure900is configured to have two-way stretch.

(F2) In some embodiments of F1, the stitch pattern further facilitates the conductive trace to expand and contract along a second axis that is perpendicular to the first axis without exceeding the fixed length of the conductive trace. For example,FIG.9Cshows that the fabric structure900is in an extended state along both the x-axis and y-axis.

(F3) In some embodiments of any of F1-F2, the stitch pattern of the fabric structure allows the fabric structure to collapse via an alternating fold, wherein the conductive trace collapses along with the fabric structure. For example,FIGS.9A-9Cillustrate how the conductive deformable fabric902collapses along with fabric structure.

(F4) In some embodiments of any of F1-F3, the fabric structure includes elastic that allows the conductive deformable fabric to return to a default state.

(F5) In some embodiments of any of F1-F4, the conductive trace is linear along the non-extendable fixed length along the first axis (e.g.,FIGS.9A-9Cshow that the conductive deformable fabric902is linear along a first axis).

(F6) In some embodiments of any of F1-F5, the stitch pattern of the fabric structure is a jersey stitch pattern (e.g., a jersey pattern stitch, such as the stiches described in reference toFIGS.6A-6H).

(F7) In some embodiments of any of F1-F6, the conductive trace is embroidered onto fabric structure (e.g.,FIG.8A-8Bshow a fabric structure (e.g., a glove800) that includes one or more portions that are made from a conductive deformable fabric).

(F8) In some embodiments of any of F1-F7, a portion of the conductive trace is configured to be attached to a neuromuscular signal sensor (e.g., an electrode (e.g., a soft electrode made of a FKM)).

(F9) In some embodiments of any of F1-F8, the conductive trace is an insulated copper magnet wire.

(F10) In some embodiments of any of F1-F9, the wearable device is machine washable.

(F11) In some embodiments of any of F1-F10, the conductive deformable fabric is configured to contract to a size that is 300 percent less than the fixed length of the conductive trace (e.g.,FIGS.9A-9Cillustrate that the fabric structure (e.g., a glove800) that includes one or more portions that are made from a conductive deformable fabric that is configured to contract to a size 300 percent less than the length of the conductive deformable fabric when fully extended).

(F12) In some embodiments of any of F1-F11, a first portion of the conductive trace is configured to be in contact with a second portion of the conductive trace and does not electrically short.

(F13) In some embodiments of any of F1-F12, the conductive deformable fabric is configured to unfold and fold in a oscillating fashion for 8,000-20,000 number of cycles without performance degradation.

(F14) In some embodiments of any of F1-F13, electrical resistivity of conductive trace is increased (or decreased) in accordance with the width of the conductive trace along the fixed length of the conductive trace (e.g., thereby allowing for pose determinations to be made based on resulting values based on the changes in resistivity). For example,FIGS.8A-9Call show how a strain value is calculated based on the changes in resistivity in accordance with the conductive deformable fabrics changes in length (e.g., unfolding).

(F15) In some embodiments of any of F1-F14, the unfolding and folding in a oscillating fashion follows an origami based folding technique.

(F16) In some embodiments of any of F1-F15, the conductive trace provides a signal that can be used to determine an amount of strain at the fabric structure (e.g., and consequently at the wearable device). For example,FIGS.8A-9Call show how a strain value is calculated based on the changes in resistivity in accordance with the conductive deformable fabrics changes in length (e.g., unfolding).

(F17) In some embodiments of F16, the amount of strain on the fabric structure is used to determine movement of joint for interacting with an artificial reality environment.FIGS.8A-8Bshow a user801wearing an artificial-reality headset103and the changes in resistivity of the glove in accordance with it unfolding can produce an input into the artificial reality environment.

Features described above in reference to A1 to F17 can be interchanged. For example any technique concerning a multi-dimensional knitting machine can be used to produce any of the knitted fabrics/garments described in reference to A1 to F17.

One of ordinary skill in the art would appreciate that the methods of use, methods of manufacturing, and devices described above can be incorporate into a single wearable device and manufacturing process of that device. For example, a knitting machine produced from the method of manufacturing a knitting machine described in reference to D1 can be used to produce a wearable device (e.g., a glove) that includes two or more of: a force sensing device described in reference to A1-B17, a knitted fabric that includes a non-knitted structure that is produced from the method of manufacturing described in reference to C1-C16, a dual density fabric described in the method described in reference to E1-E18, and/or the wearable device that comprises a conductive deformable fabric described in reference to F1-F17.

In other example embodiments, which are described in Appendix A, a wristband can be provide. The wristband can include a textile main body; a textile electrode located at a surface of the textile main body; a flexible printed circuit; and textile conductive traces electrically connecting the textile electrode with the flexible printed circuit. These textile conductive traces can be integrated with knit structures using the techniques described above, and additional details regarding this wristband are also provided in Appendix A. The textile electrode can be located along an inner surface of the textile main body. The textile electrode can include a conductive yarn (examples of which are described in Appendix A). The textile main body and the textile electrode can be formed using a method selected from the group consisting of knitting, weaving, and embroidery. The flexible printed circuit can be integrated into the textile main body.

In another aspect also described in Appendix A, a fabric electrode can be provided that includes a knit, woven, or embroidered textile.

The knitted structures described above can be implemented in various forms and can be used in conjunction with artificial-reality systems (e.g., to provide a soft wearable glove for use as an input and sensing device for use with artificial-reality systems). Thus, described below are examples of wrist-wearable devices, headset devices, systems, and haptic feedback devices to provide further context for the systems in which the techniques described herein can be utilized. Specific operations described above may occur as a result of specific hardware, such hardware is described in further detail below. The devices described below are not limiting and features on these devices can be removed or additional features can be added to these devices.

Example Wrist-Wearable Devices

FIGS.14A and14Billustrate an example wrist-wearable device1450, in accordance with some embodiments. The wrist-wearable device1450is an instance of the wearable device described herein, such that the wearable device should be understood to have the features of the wrist-wearable device1450and vice versa.FIG.14Aillustrates a perspective view of the wrist-wearable device1450that includes a watch body1454coupled with a watch band1462. The watch body1454and the watch band1462can have a substantially rectangular or circular shape and can be configured to allow a user to wear the wrist-wearable device1450on a body part (e.g., a wrist). The wrist-wearable device1450can include a retaining mechanism1467(e.g., a buckle, a hook and loop fastener, etc.) for securing the watch band1462to the user's wrist. The wrist-wearable device1450can also include a coupling mechanism1460(e.g., a cradle) for detachably coupling the capsule or watch body1454(via a coupling surface of the watch body1454) to the watch band1462.

The wrist-wearable device1450can perform various functions associated with navigating through user interfaces and selectively opening applications. As will be described in more detail below, operations executed by the wrist-wearable device1450can include, without limitation, display of visual content to the user (e.g., visual content displayed on display1456); sensing user input (e.g., sensing a touch on peripheral button1468, sensing biometric data on sensor1464, sensing neuromuscular signals on neuromuscular sensor1465, etc.); messaging (e.g., text, speech, video, etc.); image capture; wireless communications (e.g., cellular, near field, Wi-Fi, personal area network, etc.); location determination; financial transactions; providing haptic feedback; alarms; notifications; biometric authentication; health monitoring; sleep monitoring; etc. These functions can be executed independently in the watch body1454, independently in the watch band1462, and/or in communication between the watch body1454and the watch band1462. In some embodiments, functions can be executed on the wrist-wearable device1450in conjunction with an artificial-reality environment that includes, but is not limited to, virtual-reality (VR) environments (including non-immersive, semi-immersive, and fully immersive VR environments); augmented-reality environments (including marker-based augmented-reality environments, markerless augmented-reality environments, location-based augmented-reality environments, and projection-based augmented-reality environments); hybrid reality; and other types of mixed-reality environments. As the skilled artisan will appreciate upon reading the descriptions provided herein, the novel wearable devices described herein can be used with any of these types of artificial-reality environments.

The watch band1462can be configured to be worn by a user such that an inner surface of the watch band1462is in contact with the user's skin. When worn by a user, sensor1464is in contact with the user's skin. The sensor1464can be a biosensor that senses a user's heart rate, saturated oxygen level, temperature, sweat level, muscle intentions, or a combination thereof. The watch band1462can include multiple sensors1464that can be distributed on an inside and/or an outside surface of the watch band1462. Additionally, or alternatively, the watch body1454can include sensors that are the same or different than those of the watch band1462(or the watch band1462can include no sensors at all in some embodiments). For example, multiple sensors can be distributed on an inside and/or an outside surface of the watch body1454. As described below with reference toFIGS.14B and/or14C, the watch body1454can include, without limitation, a front-facing image sensor1425A and/or a rear-facing image sensor1425B, a biometric sensor, an IMU, a heart rate sensor, a saturated oxygen sensor, a neuromuscular sensor(s), an altimeter sensor, a temperature sensor, a bioimpedance sensor, a pedometer sensor, an optical sensor (e.g., imaging sensor14104), a touch sensor, a sweat sensor, etc. The sensor1464can also include a sensor that provides data about a user's environment including a user's motion (e.g., an IMU), altitude, location, orientation, gait, or a combination thereof. The sensor1464can also include a light sensor (e.g., an infrared light sensor, a visible light sensor) that is configured to track a position and/or motion of the watch body1454and/or the watch band1462. The watch band1462can transmit the data acquired by sensor1464to the watch body1454using a wired communication method (e.g., a Universal Asynchronous Receiver/Transmitter (UART), a USB transceiver, etc.) and/or a wireless communication method (e.g., near field communication, Bluetooth, etc.). The watch band1462can be configured to operate (e.g., to collect data using sensor1464) independent of whether the watch body1454is coupled to or decoupled from watch band1462.

In some examples, the watch band1462can include a neuromuscular sensor1465(e.g., an EMG sensor, a mechanomyogram (MMG) sensor, a sonomyography (SMG) sensor, etc.). Neuromuscular sensor1465can sense a user's intention to perform certain motor actions. The sensed muscle intention can be used to control certain user interfaces displayed on the display1456of the wrist-wearable device1450and/or can be transmitted to a device responsible for rendering an artificial-reality environment (e.g., a head-mounted display) to perform an action in an associated artificial-reality environment, such as to control the motion of a virtual device displayed to the user.

Signals from neuromuscular sensor1465can be used to provide a user with an enhanced interaction with a physical object and/or a virtual object in an artificial-reality application generated by an artificial-reality system (e.g., user interface objects presented on the display1456, or another computing device (e.g., a smartphone)). Signals from neuromuscular sensor1465can be obtained (e.g., sensed and recorded) by one or more neuromuscular sensors1465of the watch band1462. AlthoughFIG.14Ashows one neuromuscular sensor1465, the watch band1462can include a plurality of neuromuscular sensors1465arranged circumferentially on an inside surface of the watch band1462such that the plurality of neuromuscular sensors1465contact the skin of the user. The watch band1462can include a plurality of neuromuscular sensors1465arranged circumferentially on an inside surface of the watch band1462. Neuromuscular sensor1465can sense and record neuromuscular signals from the user as the user performs muscular activations (e.g., movements, gestures, etc.). The muscular activations performed by the user can include static gestures, such as placing the user's hand palm down on a table; dynamic gestures, such as grasping a physical or virtual object; and covert gestures that are imperceptible to another person, such as slightly tensing a joint by co-contracting opposing muscles or using sub-muscular activations. The muscular activations performed by the user can include symbolic gestures (e.g., gestures mapped to other gestures, interactions, or commands, for example, based on a gesture vocabulary that specifies the mapping of gestures to commands).

The watch band1462and/or watch body1454can include a haptic device1463(e.g., a vibratory haptic actuator) that is configured to provide haptic feedback (e.g., a cutaneous and/or kinesthetic sensation, etc.) to the user's skin. The sensors1464and1465, and/or the haptic device1463can be configured to operate in conjunction with multiple applications including, without limitation, health monitoring, social media, game playing, and artificial reality (e.g., the applications associated with artificial reality).

The wrist-wearable device1450can include a coupling mechanism (also referred to as a cradle) for detachably coupling the watch body1454to the watch band1462. A user can detach the watch body1454from the watch band1462in order to reduce the encumbrance of the wrist-wearable device1450to the user. The wrist-wearable device1450can include a coupling surface on the watch body1454and/or coupling mechanism(s)1460(e.g., a cradle, a tracker band, a support base, a clasp). A user can perform any type of motion to couple the watch body1454to the watch band1462and to decouple the watch body1454from the watch band1462. For example, a user can twist, slide, turn, push, pull, or rotate the watch body1454relative to the watch band1462, or a combination thereof, to attach the watch body1454to the watch band1462and to detach the watch body1454from the watch band1462.

As shown in the example ofFIG.14A, the watch band coupling mechanism1460can include a type of frame or shell that allows the watch body1454coupling surface to be retained within the watch band coupling mechanism1460. The watch body1454can be detachably coupled to the watch band1462through a friction fit, magnetic coupling, a rotation-based connector, a shear-pin coupler, a retention spring, one or more magnets, a clip, a pin shaft, a hook and loop fastener, or a combination thereof. In some examples, the watch body1454can be decoupled from the watch band1462by actuation of the release mechanism1470. The release mechanism1470can include, without limitation, a button, a knob, a plunger, a handle, a lever, a fastener, a clasp, a dial, a latch, or a combination thereof.

As shown inFIGS.14A-14B, the coupling mechanism1460can be configured to receive a coupling surface proximate to the bottom side of the watch body1454(e.g., a side opposite to a front side of the watch body1454where the display1456is located), such that a user can push the watch body1454downward into the coupling mechanism1460to attach the watch body1454to the coupling mechanism1460. In some embodiments, the coupling mechanism1460can be configured to receive a top side of the watch body1454(e.g., a side proximate to the front side of the watch body1454where the display1456is located) that is pushed upward into the cradle, as opposed to being pushed downward into the coupling mechanism1460. In some embodiments, the coupling mechanism1460is an integrated component of the watch band1462such that the watch band1462and the coupling mechanism1460are a single unitary structure.

The wrist-wearable device1450can include a single release mechanism1470or multiple release mechanisms1470(e.g., two release mechanisms1470positioned on opposing sides of the wrist-wearable device1450such as spring-loaded buttons). As shown inFIG.14A, the release mechanism1470can be positioned on the watch body1454and/or the watch band coupling mechanism1460. AlthoughFIG.14Ashows release mechanism1470positioned at a corner of watch body1454and at a corner of watch band coupling mechanism1460, the release mechanism1470can be positioned anywhere on watch body1454and/or watch band coupling mechanism1460that is convenient for a user of wrist-wearable device1450to actuate. A user of the wrist-wearable device1450can actuate the release mechanism1470by pushing, turning, lifting, depressing, shifting, or performing other actions on the release mechanism1470. Actuation of the release mechanism1470can release (e.g., decouple) the watch body1454from the watch band coupling mechanism1460and the watch band1462allowing the user to use the watch body1454independently from watch band1462. For example, decoupling the watch body1454from the watch band1462can allow the user to capture images using rear-facing image sensor1425B.

FIG.14Bincludes top views of examples of the wrist-wearable device1450. The examples of the wrist-wearable device1450shown inFIGS.14A-14Bcan include a coupling mechanism1460(as shown inFIG.14B, the shape of the coupling mechanism can correspond to the shape of the watch body1454of the wrist-wearable device1450). The watch body1454can be detachably coupled to the coupling mechanism1460through a friction fit, magnetic coupling, a rotation-based connector, a shear-pin coupler, a retention spring, one or more magnets, a clip, a pin shaft, a hook and loop fastener, or any combination thereof.

In some examples, the watch body1454can be decoupled from the coupling mechanism1460by actuation of a release mechanism1470. The release mechanism1470can include, without limitation, a button, a knob, a plunger, a handle, a lever, a fastener, a clasp, a dial, a latch, or a combination thereof. In some examples, the wristband system functions can be executed independently in the watch body1454, independently in the coupling mechanism1460, and/or in communication between the watch body1454and the coupling mechanism1460. The coupling mechanism1460can be configured to operate independently (e.g., execute functions independently) from watch body1454. Additionally, or alternatively, the watch body1454can be configured to operate independently (e.g., execute functions independently) from the coupling mechanism1460. As described below with reference to the block diagram ofFIG.14A, the coupling mechanism1460and/or the watch body1454can each include the independent resources required to independently execute functions. For example, the coupling mechanism1460and/or the watch body1454can each include a power source (e.g., a battery), a memory, data storage, a processor (e.g., a central processing unit (CPU)), communications, a light source, and/or input/output devices.

The wrist-wearable device1450can have various peripheral buttons1472,1474, and1476, for performing various operations at the wrist-wearable device1450. Also, various sensors, including one or both of the sensors1464and1465, can be located on the bottom of the watch body1454, and can optionally be used even when the watch body1454is detached from the watch band1462.

FIG.14Cis a block diagram of a computing system14000, according to at least one embodiment of the present disclosure. The computing system14000includes an electronic device14002, which can be, for example, a wrist-wearable device. The wrist-wearable device1450described in detail above with respect toFIGS.14A-14Bis an example of the electronic device14002, so the electronic device14002will be understood to include the components shown and described below for the computing system14000. In some embodiments, all, or a substantial portion of the components of the computing system14000are included in a single integrated circuit. In some embodiments, the computing system14000can have a split architecture (e.g., a split mechanical architecture, a split electrical architecture) between a watch body (e.g., a watch body1454inFIGS.14A-14B) and a watch band (e.g., a watch band1462inFIGS.14A-14B). The electronic device14002can include a processor (e.g., a central processing unit14004), a controller14010, a peripherals interface14014that includes one or more sensors14100and various peripheral devices, a power source (e.g., a power system14300), and memory (e.g., a memory14400) that includes an operating system (e.g., an operating system14402), data (e.g., data14410), and one or more applications (e.g., applications14430).

In some embodiments, the computing system14000includes the power system14300which includes a charger input14302, a power-management integrated circuit (PMIC)14304, and a battery14306.

In some embodiments, a watch body and a watch band can each be electronic devices14002that each have respective batteries (e.g., battery14306), and can share power with each other. The watch body and the watch band can receive a charge using a variety of techniques. In some embodiments, the watch body and the watch band can use a wired charging assembly (e.g., power cords) to receive the charge. Alternatively, or in addition, the watch body and/or the watch band can be configured for wireless charging. For example, a portable charging device can be designed to mate with a portion of watch body and/or watch band and wirelessly deliver usable power to a battery of watch body and/or watch band.

The watch body and the watch band can have independent power systems14300to enable each to operate independently. The watch body and watch band can also share power (e.g., one can charge the other) via respective PMICs14304that can share power over power and ground conductors and/or over wireless charging antennas.

In some embodiments, the peripherals interface14014can include one or more sensors14100. The sensors14100can include a coupling sensor14102for detecting when the electronic device14002is coupled with another electronic device14002(e.g., a watch body can detect when it is coupled to a watch band, and vice versa). The sensors14100can include imaging sensors14104for collecting imaging data, which can optionally be the same device as one or more of the cameras14218. In some embodiments, the imaging sensors14104can be separate from the cameras14218. In some embodiments the sensors include an SpO2 sensor14106. In some embodiments, the sensors14100include an EMG sensor14108for detecting, for example muscular movements by a user of the electronic device14002. In some embodiments, the sensors14100include a capacitive sensor14110for detecting changes in potential of a portion of a user's body. In some embodiments, the sensors14100include a heart rate sensor14112. In some embodiments, the sensors5100include an inertial measurement unit (IMU) sensor14114for detecting, for example, changes in acceleration of the user's hand.

In some embodiments, the peripherals interface14014includes a near-field communication (NFC) component14202, a global-position system (GPS) component14204, a long-term evolution (LTE) component14206, and or a Wi-Fi or Bluetooth communication component14208.

In some embodiments, the peripherals interface includes one or more buttons (e.g., the peripheral buttons1457,1458, and1459inFIG.14B), which, when selected by a user, cause operation to be performed at the electronic device14002.

The electronic device14002can include at least one display14212, for displaying visual affordances to the user, including user-interface elements and/or three-dimensional virtual objects. The display can also include a touch screen for inputting user inputs, such as touch gestures, swipe gestures, and the like.

The electronic device14002can include at least one speaker14214and at least one microphone14216for providing audio signals to the user and receiving audio input from the user. The user can provide user inputs through the microphone14216and can also receive audio output from the speaker14214as part of a haptic event provided by the haptic controller14012.

The electronic device14002can include at least one camera14218, including a front camera14220and a rear camera14222. In some embodiments, the electronic device14002can be a head-wearable device, and one of the cameras14218can be integrated with a lens assembly of the head-wearable device.

One or more of the electronic devices14002can include one or more haptic controllers14012and associated componentry for providing haptic events at one or more of the electronic devices14002(e.g., a vibrating sensation or audio output in response to an event at the electronic device14002). The haptic controllers14012can communicate with one or more electroacoustic devices, including a speaker of the one or more speakers14214and/or other audio components and/or electromechanical devices that convert energy into linear motion such as a motor, solenoid, electroactive polymer, piezoelectric actuator, electrostatic actuator, or other tactile output generating component (e.g., a component that converts electrical signals into tactile outputs on the device). The haptic controller14012can provide haptic events to that are capable of being sensed by a user of the electronic devices14002. In some embodiments, the one or more haptic controllers14012can receive input signals from an application of the applications14430.

Memory14400optionally includes high-speed random-access memory and optionally also includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to the memory14400by other components of the electronic device14002, such as the one or more processors of the central processing unit14004, and the peripherals interface14014is optionally controlled by a memory controller of the controllers14010.

In some embodiments, software components stored in the memory14400can include one or more operating systems14402(e.g., a Linux-based operating system, an Android operating system, etc.). The memory14400can also include data14410, including structured data (e.g., SQL databases, MongoDB databases, GraphQL data, JSON data, etc.). The data14410can include profile data14412, sensor data14414, media file data14414.

In some embodiments, software components stored in the memory14400include one or more applications14430configured to be perform operations at the electronic devices14002. In some embodiments, the one or more applications14430include one or more communication interface modules14432, one or more graphics modules14434, one or more camera application modules14436. In some embodiments, a plurality of applications14430can work in conjunction with one another to perform various tasks at one or more of the electronic devices14002.

It should be appreciated that the electronic devices14002are only some examples of the electronic devices14002within the computing system14000, and that other electronic devices14002that are part of the computing system14000can have more or fewer components than shown optionally combines two or more components, or optionally have a different configuration or arrangement of the components. The various components shown inFIG.14Care implemented in hardware, software, firmware, or a combination thereof, including one or more signal processing and/or application-specific integrated circuits.

As illustrated by the lower portion ofFIG.14C, various individual components of a wrist-wearable device can be examples of the electronic device14002. For example, some or all of the components shown in the electronic device14002can be housed or otherwise disposed in a combined watch device14002A, or within individual components of the capsule device watch body14002B, the cradle portion14002C, and/or a watch band.

FIG.14Dillustrates a wearable device14170, in accordance with some embodiments. In some embodiments, the wearable device14170is used to generate control information (e.g., sensed data about neuromuscular signals or instructions to perform certain commands after the data is sensed) for causing a computing device to perform one or more input commands. In some embodiments, the wearable device14170includes a plurality of neuromuscular sensors14176. In some embodiments, the plurality of neuromuscular sensors14176includes a predetermined number of (e.g.,16) neuromuscular sensors (e.g., EMG sensors) arranged circumferentially around an elastic band14174. The plurality of neuromuscular sensors14176may include any suitable number of neuromuscular sensors. In some embodiments, the number and arrangement of neuromuscular sensors14176depends on the particular application for which the wearable device14170is used. For instance, a wearable device14170configured as an armband, wristband, or chest-band may include a plurality of neuromuscular sensors14176with different number of neuromuscular sensors and different arrangement for each use case, such as medical use cases as compared to gaming or general day-to-day use cases. For example, at least 16 neuromuscular sensors14176may be arranged circumferentially around elastic band14174.

In some embodiments, the elastic band14174is configured to be worn around a user's lower arm or wrist. The elastic band14174may include a flexible electronic connector14172. In some embodiments, the flexible electronic connector14172interconnects separate sensors and electronic circuitry that are enclosed in one or more sensor housings. Alternatively, in some embodiments, the flexible electronic connector14172interconnects separate sensors and electronic circuitry that are outside of the one or more sensor housings. Each neuromuscular sensor of the plurality of neuromuscular sensors14176can include a skin-contacting surface that includes one or more electrodes. One or more sensors of the plurality of neuromuscular sensors14176can be coupled together using flexible electronics incorporated into the wearable device14170. In some embodiments, one or more sensors of the plurality of neuromuscular sensors14176can be integrated into a knitted fabric, wherein the fabric one or more sensors of the plurality of neuromuscular sensors14176are knitted into the fabric and mimic the pliability of fabric (e.g., the one or more sensors of the plurality of neuromuscular sensors14176can be constructed from a series knitted strands of yarn). In some embodiments, the sensors are flush with the surface of the textile and are indistinguishable from the textile when worn by the user.

FIG.14Eillustrates a wearable device14179in accordance with some embodiments. The wearable device14179includes paired sensor channels14185a-14185falong an interior surface of a wearable structure14175that are configured to detect neuromuscular signals. Different number of paired sensors channels can be used (e.g., one pair of sensors, three pairs of sensors, four pairs of sensors, or six pairs of sensors). The wearable structure14175can include a band portion14190, a capsule portion14195, and a cradle portion (not pictured) that is coupled with the band portion14190to allow for the capsule portion14195to be removably coupled with the band portion14190. For embodiments in which the capsule portion14195is removable, the capsule portion14195can be referred to as a removable structure, such that in these embodiments the wearable device includes a wearable portion (e.g., band portion14190and the cradle portion) and a removable structure (the removable capsule portion which can be removed from the cradle). In some embodiments, the capsule portion14195includes the one or more processors and/or other components of the wearable device1688described above in reference toFIGS.16A and16B. The wearable structure14175is configured to be worn by a user1611. More specifically, the wearable structure14175is configured to couple the wearable device14179to a wrist, arm, forearm, or other portion of the user's body. Each paired sensor channels14185a-14185fincludes two electrodes14180(e.g., electrodes14180a-14180h) for sensing neuromuscular signals based on differential sensing within each respective sensor channel. In accordance with some embodiments, the wearable device14170further includes an electrical ground and a shielding electrode.

The techniques described above can be used with any device for sensing neuromuscular signals, including the arm-wearable devices ofFIG.14A-14C, but could also be used with other types of wearable devices for sensing neuromuscular signals (such as body-wearable or head-wearable devices that might have neuromuscular sensors closer to the brain or spinal column).

In some embodiments, a wrist-wearable device can be used in conjunction with a head-wearable device described below, and the wrist-wearable device can also be configured to be used to allow a user to control aspect of the artificial reality (e.g., by using EMG-based gestures to control user interface objects in the artificial reality and/or by allowing a user to interact with the touchscreen on the wrist-wearable device to also control aspects of the artificial reality). Having thus described example wrist-wearable device, attention will now be turned to example head-wearable devices, such AR glasses and VR headsets.

Example Head-Wearable Devices

FIG.15Ashows an example AR system1500which can be controlled by using the knitted structures (e.g., wearable gloves or other wearable structures formed in accordance with the knitting techniques described herein), in accordance with some embodiments. InFIG.15A, the AR system1500includes an eyewear device with a frame1502configured to hold a left display device1506-1and a right display device1506-2in front of a user's eyes. The display devices1506-1and1506-2may act together or independently to present an image or series of images to a user. While the AR system1500includes two displays, embodiments of this disclosure may be implemented in AR systems with a single near-eye display (NED) or more than two NEDs.

In some embodiments, the AR system1500includes one or more sensors, such as the acoustic sensors1504. For example, the acoustic sensors1504can generate measurement signals in response to motion of the AR system1500and may be located on substantially any portion of the frame1502. Any one of the sensors may be a position sensor, an IMU, a depth camera assembly, or any combination thereof. In some embodiments, the AR system1500includes more or fewer sensors than are shown inFIG.15A. In embodiments in which the sensors include an IMU, the IMU may generate calibration data based on measurement signals from the sensors. Examples of the sensors include, without limitation, accelerometers, gyroscopes, magnetometers, other suitable types of sensors that detect motion, sensors used for error correction of the IMU, or some combination thereof.

In some embodiments, the AR system1500includes a microphone array with a plurality of acoustic sensors1504-1through1504-8, referred to collectively as the acoustic sensors1504. The acoustic sensors1504may be transducers that detect air pressure variations induced by sound waves. In some embodiments, each acoustic sensor1504is configured to detect sound and convert the detected sound into an electronic format (e.g., an analog or digital format). In some embodiments, the microphone array includes ten acoustic sensors:1504-1and1504-2designed to be placed inside a corresponding ear of the user, acoustic sensors1504-3,1504-4,1504-5,1504-6,1504-7, and1504-8positioned at various locations on the frame1502, and acoustic sensors positioned on a corresponding neckband, where the neckband is an optional component of the system that is not present in certain embodiments of the artificial-reality systems discussed herein.

The configuration of the acoustic sensors1504of the microphone array may vary. While the AR system1500is shown inFIG.15Ahaving ten acoustic sensors1504, the number of acoustic sensors1504may be more or fewer than ten. In some situations, using more acoustic sensors1504increases the amount of audio information collected and/or the sensitivity and accuracy of the audio information. In contrast, in some situations, using a lower number of acoustic sensors1504decreases the computing power required by a controller to process the collected audio information. In addition, the position of each acoustic sensor1504of the microphone array may vary. For example, the position of an acoustic sensor1504may include a defined position on the user, a defined coordinate on the frame1502, an orientation associated with each acoustic sensor, or some combination thereof.

The acoustic sensors1504-1and1504-2may be positioned on different parts of the user's ear. In some embodiments, there are additional acoustic sensors on or surrounding the ear in addition to acoustic sensors1504inside the ear canal. In some situations, having an acoustic sensor positioned next to an ear canal of a user enables the microphone array to collect information on how sounds arrive at the ear canal. By positioning at least two of the acoustic sensors1504on either side of a user's head (e.g., as binaural microphones), the AR device1500is able to simulate binaural hearing and capture a 3D stereo sound field around a user's head. In some embodiments, the acoustic sensors1504-1and1504-2are connected to the AR system1500via a wired connection, and in other embodiments, the acoustic sensors1504-1and1504-2are connected to the AR system1500via a wireless connection (e.g., a Bluetooth connection). In some embodiments, the AR system1500does not include the acoustic sensors1504-1and1504-2.

The acoustic sensors1504on the frame1502may be positioned along the length of the temples, across the bridge of the nose, above or below the display devices1506, or in some combination thereof. The acoustic sensors1504may be oriented such that the microphone array is able to detect sounds in a wide range of directions surrounding the user that is wearing the AR system1500. In some embodiments, a calibration process is performed during manufacturing of the AR system1500to determine relative positioning of each acoustic sensor1504in the microphone array.

In some embodiments, the eyewear device further includes, or is communicatively coupled to, an external device (e.g., a paired device), such as the optional neckband discussed above. In some embodiments, the optional neckband is coupled to the eyewear device via one or more connectors. The connectors may be wired or wireless connectors and may include electrical and/or non-electrical (e.g., structural) components. In some embodiments, the eyewear device and the neckband operate independently without any wired or wireless connection between them. In some embodiments, the components of the eyewear device and the neckband are located on one or more additional peripheral devices paired with the eyewear device, the neckband, or some combination thereof. Furthermore, the neckband is intended to represent any suitable type or form of paired device. Thus, the following discussion of neckband may also apply to various other paired devices, such as smart watches, smart phones, wrist bands, other wearable devices, hand-held controllers, tablet computers, or laptop computers.

In some situations, pairing external devices, such as the optional neckband, with the AR eyewear device enables the AR eyewear device to achieve the form factor of a pair of glasses while still providing sufficient battery and computation power for expanded capabilities. Some, or all, of the battery power, computational resources, and/or additional features of the AR system1500may be provided by a paired device or shared between a paired device and an eyewear device, thus reducing the weight, heat profile, and form factor of the eyewear device overall while still retaining desired functionality. For example, the neckband may allow components that would otherwise be included on an eyewear device to be included in the neckband thereby shifting a weight load from a user's head to a user's shoulders. In some embodiments, the neckband has a larger surface area over which to diffuse and disperse heat to the ambient environment. Thus, the neckband may allow for greater battery and computation capacity than might otherwise have been possible on a stand-alone eyewear device. Because weight carried in the neckband may be less invasive to a user than weight carried in the eyewear device, a user may tolerate wearing a lighter eyewear device and carrying or wearing the paired device for greater lengths of time than the user would tolerate wearing a heavy, stand-alone eyewear device, thereby enabling an artificial-reality environment to be incorporated more fully into a user's day-to-day activities.

In some embodiments, the optional neckband is communicatively coupled with the eyewear device and/or to other devices. The other devices may provide certain functions (e.g., tracking, localizing, depth mapping, processing, storage, etc.) to the AR system1500. In some embodiments, the neckband includes a controller and a power source. In some embodiments, the acoustic sensors of the neckband are configured to detect sound and convert the detected sound into an electronic format (analog or digital).

The controller of the neckband processes information generated by the sensors on the neckband and/or the AR system1500. For example, the controller may process information from the acoustic sensors1504. For each detected sound, the controller may perform a direction of arrival (DOA) estimation to estimate a direction from which the detected sound arrived at the microphone array. As the microphone array detects sounds, the controller may populate an audio data set with the information. In embodiments in which the AR system1500includes an IMU, the controller may compute all inertial and spatial calculations from the IMU located on the eyewear device. The connector may convey information between the eyewear device and the neckband and between the eyewear device and the controller. The information may be in the form of optical data, electrical data, wireless data, or any other transmittable data form. Moving the processing of information generated by the eyewear device to the neckband may reduce weight and heat in the eyewear device, making it more comfortable and safer for a user.

In some embodiments, the power source in the neckband provides power to the eyewear device and the neckband. The power source may include, without limitation, lithium-ion batteries, lithium-polymer batteries, primary lithium batteries, alkaline batteries, or any other form of power storage. In some embodiments, the power source is a wired power source.

As noted, some artificial-reality systems may, instead of blending an artificial reality with actual reality, substantially replace one or more of a user's sensory perceptions of the real world with a virtual experience. One example of this type of system is a head-worn display system, such as the VR system1550inFIG.15B, which mostly or completely covers a user's field of view.

FIG.15Bshows a VR system1550(e.g., also referred to herein as VR headsets or VR headset) in accordance with some embodiments. The VR system1550includes a head-mounted display (HMD)1552. The HMD1552includes a front body1556and a frame1554(e.g., a strap or band) shaped to fit around a user's head. In some embodiments, the HMD1552includes output audio transducers1558-1and1558-2, as shown inFIG.15B(e.g., transducers). In some embodiments, the front body1556and/or the frame1554includes one or more electronic elements, including one or more electronic displays, one or more IMUs, one or more tracking emitters or detectors, and/or any other suitable device or sensor for creating an artificial-reality experience.

Artificial-reality systems may include a variety of types of visual feedback mechanisms. For example, display devices in the AR system1500and/or the VR system1550may include one or more liquid-crystal displays (LCDs), light emitting diode (LED) displays, organic LED (OLED) displays, and/or any other suitable type of display screen. Artificial-reality systems may include a single display screen for both eyes or may provide a display screen for each eye, which may allow for additional flexibility for varifocal adjustments or for correcting a refractive error associated with the user's vision. Some artificial-reality systems also include optical subsystems having one or more lenses (e.g., conventional concave or convex lenses, Fresnel lenses, or adjustable liquid lenses) through which a user may view a display screen.

In addition to or instead of using display screens, some artificial-reality systems include one or more projection systems. For example, display devices in the AR system1500and/or the VR system1550may include micro-LED projectors that project light (e.g., using a waveguide) into display devices, such as clear combiner lenses that allow ambient light to pass through. The display devices may refract the projected light toward a user's pupil and may enable a user to simultaneously view both artificial-reality content and the real world. Artificial-reality systems may also be configured with any other suitable type or form of image projection system.

Artificial-reality systems may also include various types of computer vision components and subsystems. For example, the AR system1500and/or the VR system1550can include one or more optical sensors such as two-dimensional (2D) or three-dimensional (3D) cameras, time-of-flight depth sensors, single-beam or sweeping laser rangefinders, 3D LiDAR sensors, and/or any other suitable type or form of optical sensor. An artificial-reality system may process data from one or more of these sensors to identify a location of a user, to map the real world, to provide a user with context about real-world surroundings, and/or to perform a variety of other functions. For example,FIG.15Bshows VR system1550having cameras1560-1and1560-2that can be used to provide depth information for creating a voxel field and a two-dimensional mesh to provide object information to the user to avoid collisions.FIG.15Balso shows that the VR system includes one or more additional cameras1562that are configured to augment the cameras1560-1and1560-2by providing more information. For example, the additional cameras1562can be used to supply color information that is not discerned by cameras1560-1and1560-2. In some embodiments, cameras1560-1and1560-2and additional cameras1562can include an optional IR cut filter configured to remove IR light from being received at the respective camera sensors.

In some embodiments, the AR system1500and/or the VR system1550can include haptic (tactile) feedback systems, which may be incorporated into headwear, gloves, body suits, handheld controllers, environmental devices (e.g., chairs or floormats), and/or any other type of device or system, such as the wearable devices discussed herein. The haptic feedback systems may provide various types of cutaneous feedback, including vibration, force, traction, shear, texture, and/or temperature. The haptic feedback systems may also provide various types of kinesthetic feedback, such as motion and compliance. The haptic feedback may be implemented using motors, piezoelectric actuators, fluidic systems, and/or a variety of other types of feedback mechanisms. The haptic feedback systems may be implemented independently of other artificial-reality devices, within other artificial-reality devices, and/or in conjunction with other artificial-reality devices.

The techniques described above can be used with any device for interacting with an artificial-reality environment, including the head-wearable devices ofFIG.15A-15B, but could also be used with other types of wearable devices for sensing neuromuscular signals (such as body-wearable or head-wearable devices that might have neuromuscular sensors closer to the brain or spinal column). Having thus described example wrist-wearable device and head-wearable devices, attention will now be turned to example feedback systems that can be integrated into the devices described above or be a separate device.

Example Feedback Devices

FIG.17is a schematic showing additional components e.g., additional components to allow for providing haptic feedback using aspects of the knitted structures described herein) that can be used with the artificial-reality system1600ofFIG.16AandFIG.16B, in accordance with some embodiments. The components inFIG.17are illustrated in a particular arrangement for ease of illustration and one skilled in the art will appreciate that other arrangements are possible. Moreover, while some example features are illustrated, various other features have not been illustrated for the sake of brevity and so as not to obscure pertinent aspects of the example implementations disclosed herein.

The artificial-reality system1600may also provide feedback to the user that the action was performed. The provided feedback may be visual via the electronic display in the head-mounted display1611(e.g., displaying the simulated hand as it picks up and lifts the virtual coffee mug) and/or haptic feedback via the haptic assembly1722in the device1720. For example, the haptic feedback may prevent (or, at a minimum, hinder/resist movement of) one or more of the user's fingers from curling past a certain point to simulate the sensation of touching a solid coffee mug. To do this, the device1720changes (either directly or indirectly) a pressurized state of one or more of the haptic assemblies1722. Each of the haptic assemblies1722includes a mechanism that, at a minimum, provides resistance when the respective haptic assembly1722is transitioned from a first pressurized state (e.g., atmospheric pressure or deflated) to a second pressurized state (e.g., inflated to a threshold pressure). Structures of haptic assemblies1722can be integrated into various devices configured to be in contact or proximity to a user's skin, including, but not limited to devices such as glove worn devices, body worn clothing device, headset devices (e.g., artificial-reality headset803inFIGS.8A-8B).

As noted above, the haptic assemblies1722described herein are configured to transition between a first pressurized state and a second pressurized state to provide haptic feedback to the user. Due to the ever-changing nature of artificial reality, the haptic assemblies1722may be required to transition between the two states hundreds, or perhaps thousands of times, during a single use. Thus, the haptic assemblies1722described herein are durable and designed to quickly transition from state to state. To provide some context, in the first pressurized state, the haptic assemblies1722do not impede free movement of a portion of the wearer's body. For example, one or more haptic assemblies1722incorporated into a glove are made from flexible materials that do not impede free movement of the wearer's hand and fingers (e.g., an electrostatic-zipping actuator). The haptic assemblies1722are configured to conform to a shape of the portion of the wearer's body when in the first pressurized state. However, once in the second pressurized state, the haptic assemblies1722are configured to impede free movement of the portion of the wearer's body. For example, the respective haptic assembly1722(or multiple respective haptic assemblies) can restrict movement of a wearer's finger (e.g., prevent the finger from curling or extending) when the haptic assembly1722is in the second pressurized state. Moreover, once in the second pressurized state, the haptic assemblies1722may take different shapes, with some haptic assemblies1722configured to take a planar, rigid shape (e.g., flat and rigid), while some other haptic assemblies1722are configured to curve or bend, at least partially.

As a non-limiting example, the system17includes a plurality of devices1720-A,1720-B, . . .1720-N, each of which includes a garment1702and one or more haptic assemblies1722(e.g., haptic assemblies1722-A,1722-B, . . . ,1722-N). As explained above, the haptic assemblies1722are configured to provide haptic stimulations to a wearer of the device1720. The garment1702of each device1720can be various articles of clothing (e.g., gloves, socks, shirts, or pants), and thus, the user may wear multiple devices1720that provide haptic stimulations to different parts of the body. Each haptic assembly1722is coupled to (e.g., embedded in or attached to) the garment1702. Further, each haptic assembly1722includes a support structure1704and at least one bladder1706. The bladder1706(e.g., a membrane) is a sealed, inflatable pocket made from a durable and puncture resistance material, such as thermoplastic polyurethane (TPU), a flexible polymer, or the like. The bladder1706contains a medium (e.g., a fluid such as air, inert gas, or even a liquid) that can be added to or removed from the bladder1706to change a pressure (e.g., fluid pressure) inside the bladder1706. The support structure1704is made from a material that is stronger and stiffer than the material of the bladder1706. A respective support structure1704coupled to a respective bladder1706is configured to reinforce the respective bladder1706as the respective bladder changes shape and size due to changes in pressure (e.g., fluid pressure) inside the bladder.

The system1700also includes a controller1714and a pressure-changing device1710. In some embodiments, the controller1714is part of the computer system1730(e.g., the processor of the computer system1730). The controller1714is configured to control operation of the pressure-changing device1710, and in turn operation of the devices1720. For example, the controller1714sends one or more signals to the pressure-changing device1710to activate the pressure-changing device1710(e.g., turn it on and off). The one or more signals may specify a desired pressure (e.g., pounds-per-square inch) to be output by the pressure-changing device1710. Generation of the one or more signals, and in turn the pressure output by the pressure-changing device1710, may be based on information collected by sensors1625inFIGS.16A and16B. For example, the one or more signals may cause the pressure-changing device1710to increase the pressure (e.g., fluid pressure) inside a first haptic assembly1722at a first time, based on the information collected by the sensors1625inFIGS.16A and16B(e.g., the user makes contact with the artificial coffee mug). Then, the controller may send one or more additional signals to the pressure-changing device1710that cause the pressure-changing device1710to further increase the pressure inside the first haptic assembly1722at a second time after the first time, based on additional information collected by the sensors1714and/or sensors1724(e.g., the user grasps and lifts the artificial coffee mug). Further, the one or more signals may cause the pressure-changing device1710to inflate one or more bladders1706in a first device1720-A, while one or more bladders1706in a second device1720-B remain unchanged. Additionally, the one or more signals may cause the pressure-changing device1710to inflate one or more bladders1706in a first device1720-A to a first pressure and inflate one or more other bladders1706in the first device1720-A to a second pressure different from the first pressure. Depending on the number of devices1720serviced by the pressure-changing device1710, and the number of bladders therein, many different inflation configurations can be achieved through the one or more signals and the examples above are not meant to be limiting.

The system1700may include an optional manifold1712between the pressure-changing device1710and the devices1720. The manifold1712may include one or more valves (not shown) that pneumatically couple each of the haptic assemblies1722with the pressure-changing device1710via tubing1708. In some embodiments, the manifold1712is in communication with the controller1714, and the controller1714controls the one or more valves of the manifold1712(e.g., the controller generates one or more control signals). The manifold1712is configured to switchably couple the pressure-changing device1710with one or more haptic assemblies1722of the same or different devices1720based on one or more control signals from the controller1714. In some embodiments, instead of using the manifold1712to pneumatically couple the pressure-changing device1710with the haptic assemblies1722, the system1700may include multiple pressure-changing devices1710, where each pressure-changing device1710is pneumatically coupled directly with a single (or multiple) haptic assembly1722. In some embodiments, the pressure-changing device1710and the optional manifold1712can be configured as part of one or more of the devices1720(not illustrated) while, in other embodiments, the pressure-changing device1710and the optional manifold1712can be configured as external to the device1720. A single pressure-changing device1710may be shared by multiple devices1720.

In some embodiments, the pressure-changing device1710is a pneumatic device, hydraulic device, a pneudraulic device, or some other device capable of adding and removing a medium (e.g., fluid, liquid, gas) from the one or more haptic assemblies1722.

The devices shown inFIG.17may be coupled via a wired connection (e.g., via busing1709). Alternatively, one or more of the devices shown inFIG.17may be wirelessly connected (e.g., via short-range communication signals). Having thus described example wrist-wearable device, example head-wearable devices, and example feedback devices, attention will now be turned to example systems that integrate one or more of the devices described above.

Example Systems

FIGS.16A and16Bare block diagrams illustrating an example artificial-reality system in accordance with some embodiments. The system1600includes one or more devices for facilitating an interactivity with an artificial-reality environment in accordance with some embodiments. For example, the head-wearable device1611can present to the user16015with a user interface within the artificial-reality environment. As a non-limiting example, the system1600includes one or more wearable devices, which can be used in conjunction with one or more computing devices. In some embodiments, the system1600provides the functionality of a virtual-reality device, an augmented-reality device, a mixed-reality device, hybrid-reality device, or a combination thereof. In some embodiments, the system1600provides the functionality of a user interface and/or one or more user applications (e.g., games, word processors, messaging applications, calendars, clocks, etc.).

The system1600can include one or more of servers1670, electronic devices1674(e.g., a computer,1674a, a smartphone1674b, a controller1674c, and/or other devices), head-wearable devices1611(e.g., the AR system1500or the VR system1550), and/or wrist-wearable devices1688(e.g., the wrist-wearable device16020). In some embodiments, the one or more of servers1670, electronic devices1674, head-wearable devices1611, and/or wrist-wearable devices1688are communicatively coupled via a network1672. In some embodiments, the head-wearable device1611is configured to cause one or more operations to be performed by a communicatively coupled wrist-wearable device1688, and/or the two devices can also both be connected to an intermediary device, such as a smartphone1674b, a controller1674c, or other device that provides instructions and data to and between the two devices. In some embodiments, the head-wearable device1611is configured to cause one or more operations to be performed by multiple devices in conjunction with the wrist-wearable device1688. In some embodiments, instructions to cause the performance of one or more operations are controlled via an artificial-reality processing module1645. The artificial-reality processing module1645can be implemented in one or more devices, such as the one or more of servers1670, electronic devices1674, head-wearable devices1611, and/or wrist-wearable devices1688. In some embodiments, the one or more devices perform operations of the artificial-reality processing module1645, using one or more respective processors, individually or in conjunction with at least one other device as described herein. In some embodiments, the system1600includes other wearable devices not shown inFIG.16AandFIG.16B, such as rings, collars, anklets, gloves, and the like.

In some embodiments, the system1600provides the functionality to control or provide commands to the one or more computing devices1674based on a wearable device (e.g., head-wearable device1611or wrist-wearable device1688) determining motor actions or intended motor actions of the user. A motor action is an intended motor action when before the user performs the motor action or before the user completes the motor action, the detected neuromuscular signals travelling through the neuromuscular pathways can be determined to be the motor action. Motor actions can be detected based on the detected neuromuscular signals, but can additionally (using a fusion of the various sensor inputs), or alternatively, be detected using other types of sensors (such as cameras focused on viewing hand movements and/or using data from an inertial measurement unit that can detect characteristic vibration sequences or other data types to correspond to particular in-air hand gestures). The one or more computing devices include one or more of a head-mounted display, smartphones, tablets, smart watches, laptops, computer systems, augmented reality systems, robots, vehicles, virtual avatars, user interfaces, a wrist-wearable device, and/or other electronic devices and/or control interfaces.

In some embodiments, the motor actions include digit movements, hand movements, wrist movements, arm movements, pinch gestures, index finger movements, middle finger movements, ring finger movements, little finger movements, thumb movements, hand clenches (or fists), waving motions, and/or other movements of the user's hand or arm.

In some embodiments, the user can define one or more gestures using the learning module. In some embodiments, the user can enter a training phase in which a user defined gesture is associated with one or more input commands that when provided to a computing device cause the computing device to perform an action. Similarly, the one or more input commands associated with the user-defined gesture can be used to cause a wearable device to perform one or more actions locally. The user-defined gesture, once trained, is stored in the memory1660. Similar to the motor actions, the one or more processors1650can use the detected neuromuscular signals by the one or more sensors1625to determine that a user-defined gesture was performed by the user.

The electronic devices1674can also include a communication interface1615, an interface1620(e.g., including one or more displays, lights, speakers, and haptic generators), one or more sensors1625, one or more applications1635, an artificial-reality processing module1645, one or more processors1650, and memory1660. The electronic devices1674are configured to communicatively couple with the wrist-wearable device1688and/or head-wearable device1611(or other devices) using the communication interface1615. In some embodiments, the electronic devices1674are configured to communicatively couple with the wrist-wearable device1688and/or head-wearable device1611(or other devices) via an application programming interface (API). In some embodiments, the electronic devices1674operate in conjunction with the wrist-wearable device1688and/or the head-wearable device1611to determine a hand gesture and cause the performance of an operation or action at a communicatively coupled device.

The server1670includes a communication interface1615, one or more applications1635, an artificial-reality processing module1645, one or more processors1650, and memory1660. In some embodiments, the server1670is configured to receive sensor data from one or more devices, such as the head-wearable device1611, the wrist-wearable device1688, and/or electronic device1674, and use the received sensor data to identify a gesture or user input. The server1670can generate instructions that cause the performance of operations and actions associated with a determined gesture or user input at communicatively coupled devices, such as the head-wearable device1611.

The head-wearable device1611includes smart glasses (e.g., the augmented-reality glasses), artificial-reality headsets (e.g., VR/AR headsets), or other head worn device. In some embodiments, one or more components of the head-wearable device1611are housed within a body of the HMD1614(e.g., frames of smart glasses, a body of a AR headset, etc.). In some embodiments, one or more components of the head-wearable device1611are stored within or coupled with lenses of the HMD1614. Alternatively or in addition, in some embodiments, one or more components of the head-wearable device1611are housed within a modular housing1606. The head-wearable device1611is configured to communicatively couple with other electronic device1674and/or a server1670using communication interface1615as discussed above.

FIG.16Bdescribes additional details of the HMD1614and modular housing1606described above in reference to16A, in accordance with some embodiments.

The housing1606include(s) a communication interface1615, circuitry1646, a power source1607(e.g., a battery for powering one or more electronic components of the housing1606and/or providing usable power to the HMD1614), one or more processors1650, and memory1660. In some embodiments, the housing1606can include one or more supplemental components that add to the functionality of the HMD1614. For example, in some embodiments, the housing1606can include one or more sensors1625, an AR processing module1645, one or more haptic generators1621, one or more imaging devices1655, one or more microphones1613, one or more speakers1617, etc. The housing1606is configured to couple with the HMD1614via the one or more retractable side straps. More specifically, the housing1606is a modular portion of the head-wearable device1611that can be removed from head-wearable device1611and replaced with another housing (which includes more or less functionality). The modularity of the housing1606allows a user to adjust the functionality of the head-wearable device1611based on their needs.

In some embodiments, the communications interface1615is configured to communicatively couple the housing1606with the HMD1614, the server1670, and/or other electronic device1674(e.g., the controller1674c, a tablet, a computer, etc.). The communication interface1615is used to establish wired or wireless connections between the housing1606and the other devices. In some embodiments, the communication interface1615includes hardware capable of data communications using any of a variety of custom or standard wireless protocols (e.g., IEEE 802.15.4, Wi-Fi, ZigBee, 6LoWPAN, Thread, Z-Wave, Bluetooth Smart, ISA100.11a, WirelessHART, or MiWi), custom or standard wired protocols (e.g., Ethernet or HomePlug), and/or any other suitable communication protocol. In some embodiments, the housing1606is configured to communicatively couple with the HMD1614and/or other electronic device1674via an application programming interface (API).

In some embodiments, the power source1607is a battery. The power source1607can be a primary or secondary battery source for the HMD1614. In some embodiments, the power source1607provides useable power to the one or more electrical components of the housing1606or the HMD1614. For example, the power source1607can provide usable power to the sensors1621, the speakers1617, the HMD1614, and the microphone1613. In some embodiments, the power source1607is a rechargeable battery. In some embodiments, the power source1607is a modular battery that can be removed and replaced with a fully charged battery while it is charged separately.

The one or more sensors1625can include heart rate sensors, neuromuscular-signal sensors (e.g., electromyography (EMG) sensors), SpO2 sensors, altimeters, thermal sensors or thermal couples, ambient light sensors, ambient noise sensors, and/or inertial measurement units (IMU)s. Additional non-limiting examples of the one or more sensors1625include, e.g., infrared, pyroelectric, ultrasonic, microphone, laser, optical, Doppler, gyro, accelerometer, resonant LC sensors, capacitive sensors, acoustic sensors, and/or inductive sensors. In some embodiments, the one or more sensors1625are configured to gather additional data about the user (e.g., an impedance of the user's body). Examples of sensor data output by these sensors includes body temperature data, infrared range-finder data, positional information, motion data, activity recognition data, silhouette detection and recognition data, gesture data, heart rate data, and other wearable device data (e.g., biometric readings and output, accelerometer data). The one or more sensors1625can include location sensing devices (e.g., GPS) configured to provide location information. In some embodiment, the data measured or sensed by the one or more sensors1625is stored in memory1660. In some embodiments, the housing1606receives sensor data from communicatively coupled devices, such as the HMD1614, the server1670, and/or other electronic device1674. Alternatively, the housing1606can provide sensors data to the HMD1614, the server1670, and/or other electronic device1674.

The one or more haptic generators1621can include one or more actuators (e.g., eccentric rotating mass (ERM), linear resonant actuators (LRA), voice coil motor (VCM), piezo haptic actuator, thermoelectric devices, solenoid actuators, ultrasonic transducers or sensors, etc.). In some embodiments, the one or more haptic generators1621are hydraulic, pneumatic, electric, and/or mechanical actuators. In some embodiments, the one or more haptic generators1621are part of a surface of the housing1606that can be used to generate a haptic response (e.g., a thermal change at the surface, a tightening or loosening of a band, increase or decrease in pressure, etc.). For example, the one or more haptic generators1625can apply vibration stimulations, pressure stimulations, squeeze simulations, shear stimulations, temperature changes, or some combination thereof to the user. In addition, in some embodiments, the one or more haptic generators1621include audio generating devices (e.g., speakers1617and other sound transducers) and illuminating devices (e.g., light-emitting diodes (LED)s, screen displays, etc.). The one or more haptic generators1621can be used to generate different audible sounds and/or visible lights that are provided to the user as haptic responses. The above list of haptic generators is non-exhaustive; any affective devices can be used to generate one or more haptic responses that are delivered to a user.

In some embodiments, the one or more applications1635include social-media applications, banking applications, health applications, messaging applications, web browsers, gaming application, streaming applications, media applications, imaging applications, productivity applications, social applications, etc. In some embodiments, the one or more applications1635include artificial reality applications. The one or more applications1635are configured to provide data to the head-wearable device1611for performing one or more operations. In some embodiments, the one or more applications1635can be displayed via a display1630of the head-wearable device1611(e.g., via the HMD1614).

In some embodiments, instructions to cause the performance of one or more operations are controlled via an artificial reality (AR) processing module1645. The AR processing module1645can be implemented in one or more devices, such as the one or more of servers1670, electronic devices1674, head-wearable devices1611, and/or wrist-wearable devices1670. In some embodiments, the one or more devices perform operations of the AR processing module1645, using one or more respective processors, individually or in conjunction with at least one other device as described herein. In some embodiments, the AR processing module1645is configured process signals based at least on sensor data. In some embodiments, the AR processing module1645is configured process signals based on image data received that captures at least a portion of the user hand, mouth, facial expression, surrounding, etc. For example, the housing1606can receive EMG data and/or IMU data from one or more sensors1625and provide the sensor data to the AR processing module1645for a particular operation (e.g., gesture recognition, facial recognition, etc.). The AR processing module1645, causes a device communicatively coupled to the housing1606to perform an operation (or action). In some embodiments, the AR processing module1645performs different operations based on the sensor data and/or performs one or more actions based on the sensor data.

In some embodiments, the one or more imaging devices1655can include an ultra-wide camera, a wide camera, a telephoto camera, a depth-sensing cameras, or other types of cameras. In some embodiments, the one or more imaging devices1655are used to capture image data and/or video data. The imaging devices1655can be coupled to a portion of the housing1606. The captured image data can be processed and stored in memory and then presented to a user for viewing. The one or more imaging devices1655can include one or more modes for capturing image data or video data. For example, these modes can include a high-dynamic range (HDR) image capture mode, a low light image capture mode, burst image capture mode, and other modes. In some embodiments, a particular mode is automatically selected based on the environment (e.g., lighting, movement of the device, etc.). For example, a wrist-wearable device with HDR image capture mode and a low light image capture mode active can automatically select the appropriate mode based on the environment (e.g., dark lighting may result in the use of low light image capture mode instead of HDR image capture mode). In some embodiments, the user can select the mode. The image data and/or video data captured by the one or more imaging devices1655is stored in memory1660(which can include volatile and non-volatile memory such that the image data and/or video data can be temporarily or permanently stored, as needed depending on the circumstances).

The circuitry1646is configured to facilitate the interaction between the housing1606and the HMD1614. In some embodiments, the circuitry1646is configured to regulate the distribution of power between the power source1607and the HMD1614. In some embodiments, the circuitry746is configured to transfer audio and/or video data between the HMD1614and/or one or more components of the housing1606.

The one or more processors1650can be implemented as any kind of computing device, such as an integrated system-on-a-chip, a microcontroller, a fixed programmable gate array (FPGA), a microprocessor, and/or other application specific integrated circuits (ASICs). The processor may operate in conjunction with memory1660. The memory1660may be or include random access memory (RAM), read-only memory (ROM), dynamic random access memory (DRAM), static random access memory (SRAM) and magnetoresistive random access memory (MRAM), and may include firmware, such as static data or fixed instructions, basic input/output system (BIOS), system functions, configuration data, and other routines used during the operation of the housing and the processor1650. The memory1660also provides a storage area for data and instructions associated with applications and data handled by the processor1650.

In some embodiments, the memory1660stores at least user data1661including sensor data1662and AR processing data1664. The sensor data1662includes sensor data monitored by one or more sensors1625of the housing1606and/or sensor data received from one or more devices communicative coupled with the housing1606, such as the HMD1614, the smartphone1674b, the controller1674c, etc. The sensor data1662can include sensor data collected over a predetermined period of time that can be used by the AR processing module1645. The AR processing data1664can include one or more one or more predefined camera-control gestures, user defined camera-control gestures, predefined non-camera-control gestures, and/or user defined non-camera-control gestures. In some embodiments, the AR processing data1664further includes one or more predetermined threshold for different gestures.

The HMD1614includes a communication interface1615, a display1630, an AR processing module1645, one or more processors, and memory. In some embodiments, the HMD1614includes one or more sensors1625, one or more haptic generators1621, one or more imaging devices1655(e.g., a camera), microphones1613, speakers1617, and/or one or more applications1635. The HMD1614operates in conjunction with the housing1606to perform one or more operations of a head-wearable device1611, such as capturing camera data, presenting a representation of the image data at a coupled display, operating one or more applications1635, and/or allowing a user to participate in an AR environment.

Any data collection performed by the devices described herein and/or any devices configured to perform or cause the performance of the different embodiments described above in reference to any of the Figures, hereinafter the “devices,” is done with user consent and in a manner that is consistent with all applicable privacy laws. Users are given options to allow the devices to collect data, as well as the option to limit or deny collection of data by the devices. A user is able to opt-in or opt-out of any data collection at any time. Further, users are given the option to request the removal of any collected data.

It will be understood that, although the terms “first,” “second,” etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the claims. As used in the description of the embodiments and the appended claims, the singular forms “a,” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term “and/or” as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

As used herein, the term “if” can be construed to mean “when” or “upon” or “in response to determining” or “in accordance with a determination” or “in response to detecting,” that a stated condition precedent is true, depending on the context. Similarly, the phrase “if it is determined [that a stated condition precedent is true]” or “if [a stated condition precedent is true]” or “when [a stated condition precedent is true]” can be construed to mean “upon determining” or “in response to determining” or “in accordance with a determination” or “upon detecting” or “in response to detecting” that the stated condition precedent is true, depending on the context.

The foregoing description, for purpose of explanation, has been described with reference to specific embodiments. However, the illustrative discussions above are not intended to be exhaustive or to limit the claims to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain principles of operation and practical applications, to thereby enable others skilled in the art.