Patent Publication Number: US-10313781-B2

Title: Audio accelerometric feedback through bilateral ear worn device system and method

Description:
PRIORITY STATEMENT 
     This application is a continuation of and claims priority to U.S. patent application Ser. No. 15/481,623 filed on Apr. 7, 2017 titled Audio Accelerometric Feedback through Bilateral Ear Worn Device System and Method which claims priority to U.S. Provisional Patent Application No. 62/320,147 filed on Apr. 8, 2016 titled Audio Accelerometric Feedback through Bilateral Ear Worn Device System and Method all of which are hereby incorporated by reference in their entireties. 
    
    
     FIELD OF THE INVENTION 
     The present invention relates to wearable devices. More particularly, but not exclusively, the present invention relates to earpieces. 
     BACKGROUND 
     Individuals who regularly engage in physical activities, such as athletes, require constant and significant feedback from many points of their bodies to properly function. Cephalic kinematics is one important area for proper body coordination and placement while performing physical activities. In fact, some believe head movement may be the most important factor in coordinating proper body placement during physical activity. However, no system or device exists optimizing head placement to maximize body coordination. 
     SUMMARY 
     In one implementation, a system includes at least one earpiece comprising at least one speaker disposed within the at least one earpiece, at least one sensor disposed within the at least one earpiece, and at least one processor operatively connected to the speakers and the sensors, wherein the at least one processor is configured to calculate a desired position of a user&#39;s head based on at least one movement reading of the user&#39;s head from the at least one sensor and to transmit a first audio signal derived from the at least one movement reading to the at least one speaker, wherein the first audio signal guides a user to the desired position of the user&#39;s head. 
     One or more of the following features may be included. The earpieces may consist of a set of earpieces configured to transmit the first audio signal to a user&#39;s tympanic membranes. The at least one processor may be configured to calculate a desired movement of the user&#39;s head based upon the at least one movement reading of the user&#39;s head from the at least one sensor and to transmit a second audio signal containing instructions derived from the at least one movement reading to the at least one speaker, wherein the second audio signal guides the user as to the desired movement of the user&#39;s head. In addition, the at least one processor may be configured to calculate the desired position of a user&#39;s head based on at least one external movement reading of another area of the user&#39;s body from at least one external sensor and to transmit a third audio signal containing instructions derived from the at least one movement reading and the at least one external movement reading to the at least one speaker, wherein the third audio signal guides the user as to the desired position of the user&#39;s head. Also, the first, second, and third audio signals may be calculated from a zero-audio reference point. 
     In another implementation, a method includes determining, via at least one sensor disposed within at least one earpiece, a current orientation of a user&#39;s head based on at least one movement reading of a user&#39;s head from the at least one sensor. At least one processor operatively connected to the at least one earpiece calculates a desired position of the user&#39;s head based on the at least one movement reading of the user&#39;s head. The at least one processor creates a first audio signal containing instructions derived from the at least one movement reading of the user&#39;s head to guide a user to the desired position of the user&#39;s head. The at least one processor then transmits the first audio signal to at least one speaker disposed within the at least one earpiece. The at least one speaker then transmits the first audio signal to at least one of the user&#39;s ears. 
     One or more of the following may be included. The at least one earpiece may comprise a set of earpieces configured to transmit the first audio signal to a user&#39;s tympanic membranes. The at least one processor may further calculate a desired movement of the user&#39;s head based on the at least one movement reading of the user&#39;s head and create a second audio signal containing instructions derived from the at least one movement reading of the user&#39;s head to guide the user as to the desired movement of the user&#39;s head. The at least one processor may then transmit the second audio signal to the at least one speaker, which then transmits the second audio signal to the at least one of the user&#39;s ears. In addition, the at least one processor may calculate the desired position of a user&#39;s head based on at least one external movement reading of another area of the user&#39;s body from at least one external sensor, create a third audio signal containing instructions derived from the at least one movement reading and the at least one external movement reading and to transmit the third audio signal to the at least one speaker, wherein the third audio signal guides the user to the desired position of the user&#39;s head. Also, the first, second, and third audio signals may be calculated from a zero-audio reference point. 
     In another implementation, a system includes a left earpiece comprising at least one microphone and a speaker and a right earpiece comprising at least one microphone and a speaker, wherein the system is configured to assist in placement of a head of a user within a three-dimensional space. In another implementation, a method to assist a user in placement of a head of the user includes providing a set of earpieces comprising a left earpiece and a right earpiece, wherein the left earpiece and the right earpiece comprise at least one microphone and a speaker, determining an orientation of the head of the user using the set of earpieces, and providing audio feedback to the user through the set of earpieces to assist the user in placement of the head of the user. 
     According to another aspect an earpiece includes an earpiece housing, a speaker, an inertial sensor disposed within the earpiece housing, and at least one processor disposed within the earpiece housing and operatively connected to the speaker and the inertial sensor. The earpiece further includes a task profile stored on a non-transitory memory of the earpiece, wherein the task profile defines head orientation or movement associated with performing a task. The at least one processor is configured to compare measurements sensed with the inertial sensor with the task profile, and if the measurements sensed exceed preset thresholds, generate an audible signal at the speaker instructing a user of the earpiece to alter head position. The earpiece may further include a transceiver disposed within the earpiece housing and operatively connected to the at least one processor for communicating with at least one external sensor operatively connected to the user, wherein the transceiver receives kinematic information from the at least one external sensor. The task profile may be used to define orientation or movement associated with performing the task for at least one of a neck of the user, a set of shoulders of the user, a torso of a user, a set of hips of the user, and a set of feet of the user. The at least one processor may be further configured to compare measurements sensed with the at least one external sensor with the task profile and if the measurements sensed with the at least one external sensor exceed preset thresholds and generate an audible signal at the speaker instructing the user of the earpiece to alter body position. The task may be associated with a sports task, a work task, or other task. 
     According to another aspect, a method for improving performance of a task using an earpiece is provided. The method includes sensing head orientation using an inertial sensor of the earpiece and evaluating by a processor of the earpiece the head orientation to determine if the head orientation exceeds one or more thresholds for performance of the task. If the head orientation or the movement exceeds one or more thresholds for performance of the task, then the method includes generating an audio signal to guide the user to alter the head orientation and transducing the audio signal at a speaker of the earpiece to guide the user to alter the head orientation as the user performs the task. The method may further include receiving a selection of the task from the user using a gestural user interface of the earpiece. The step of evaluating may include obtaining the one or more thresholds for performance of the task from a task profile stored on a non-transitory storage medium. 
     One or more of these and/or other objects, features, or advantages of the present invention will become apparent from the specification and following claims. No single embodiment need provide every object, feature, or advantage. Different embodiments may have different objects, features, or advantages. Therefore, the present invention is not to be limited to or by any objects, features, or advantages stated herein. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  illustrates a system which includes a left earpiece and a right earpiece. 
         FIG. 2  illustrates a side view of a preferred embodiment of one of the earpieces and its relationship with a user&#39;s ear 
         FIG. 3  illustrates a block diagram of the system. 
         FIG. 4  includes a flowchart of one implementation of the method. 
     
    
    
     DETAILED DESCRIPTION 
       FIG. 1  illustrates a system  10  which includes a left earpiece  12 A and a right earpiece  12 B. The left earpiece  12 A has a left earpiece housing  14 A. The right earpiece  12 B has a right earpiece housing  14 B. The left earpiece  12 A and the right earpiece  12 B as shown are configured to be placed within the external auditory canals of a user in an ear bud style configuration. However, other configurations of earpieces are contemplated including behind-the-ear configurations and head sets, and completely-in-the canal configurations. In the configuration shown, the earpieces  12 A,  12 B fit to physically block the external auditory canal, however other configurations may be used including those in which the canal remains fully open or partially open. An external microphone  22 A is shown on the left earpiece  12 A and an external microphone  22 B is shown on the right earpiece  12 B the external microphones  22 A,  22 B may be positioned to detect ambient sounds within the environment. The earpieces may include one or more sensors used to determine head position of the user and then to communicate audio signals  24 A and  24 B respectively to guide a user as to a desired placement of a user&#39;s head. The audio signals  24 A and  24 B may contain verbal instructions, sounds representing specific instructions the user can follow, or sounds stimulating certain areas of the user&#39;s ear canal or the user&#39;s tympanic membranes which can guide the user to the best placement of the user&#39;s head. One application for the system  10  is for use by athletes in sports requiring precise coordination such as gymnastics and other similar Olympic sports, soccer, basketball, swimming, skiing, martial arts, and many others. Other applications may include tactical training, marksmanship, training operation of a vehicle or machine, and physical therapy. Of course, any number of other applications are contemplated where it is desirable to give a user feedback regarding placement of the user&#39;s head. 
       FIG. 2  illustrates a side view of a right earpiece  12 B and its relationship to a user&#39;s ear. The earpiece  12 B is configured to fit comfortably within a user&#39;s ear canal  26  to both minimize the amount of external sound reaching the user&#39;s ear canal  26  and to facilitate the transmission of the audio signal  24 B from the speaker  16  to a user&#39;s tympanic membrane  28 . The earpiece  12 B may be configured to be of any size necessary to fit within the user&#39;s ear canal  26  and sleeves may be used to assist with the fit or the earpiece may be custom molded. The earpiece  12 B may also have a gesture control interface  30  in which the user can control the system  10 . The gestures used to control the system  10  include touching, tapping, swiping, use of an instrument, or any combination of the gestures. Touching gestures used to control the system  10  may be of any duration and may include the touching of areas not part of the gesture control interface  30 . Tapping gestures used to control the system  10  may include any number of taps and need not be brief. Swiping gestures used to control the system  10  may include a single swipe, a swipe changing direction at least once, a swipe with a time delay, a plurality of swipes, or any combination. The gestural control interface  30  may be performed in various ways such as using light emitters and detections, capacitive sensing, or other types of emitters and detectors. The gestural control interface  30  may be used by an individual to specify the type of activity or task the user is performing or whether the user would like to receive feedback on head or body position or movement while performing one or more tasks or activities. 
       FIG. 3  is a block diagram of the system  10  which comprises at least one earpiece  12 , at least one speaker  16  disposed within the earpiece  12 , at least one sensor  18  with at least one air microphone  42 , at least one bone conduction microphone  44 , and a plurality of inertial sensors  46 ,  48  operatively connected to the earpiece  12 , at least one microphone  22  operatively connected to the earpiece  12 , a gesture control interface  30  with at least one emitter  38  and at least one detector  40  operatively connected to the earpiece  12 , a transceiver  32  disposed within the earpiece  12 , a radio transceiver  34  disposed within the earpiece  12 , at least one LED disposed within the earpiece  12 , a battery  50  disposed within the earpiece  12  and operatively connected to all of the aforementioned components, and at least one processor  20  disposed within the earpiece  12  and operatively connected to all of the aforementioned components. The earpiece  12  may be composed of metal, plastic, composites, combinations thereof, or any other material suitable for human use and may be configured to both minimize the direct passage of ambient noise from the environment and to facilitate the reproduction of audio signals at a user&#39;s ear. The earpiece  12  may also be constructed to be waterproof. 
     One or more speakers  16  disposed within the earpiece  12  may be positioned proximate to a user&#39;s tympanic membrane and be configured to receive at least one audio signal from the one or more processors  20  and to transmit one or more audio signals while the earpiece is functional. 
     One or more sensors  18  operatively connected to the earpiece  12  may be configured to detect a user&#39;s cephalic movements and determine a current orientation of a user&#39;s head. The sensors  18  may also be configured to detect movements of other areas of a user&#39;s body, clothing or other objects worn by a user, a surface the user is standing on, or an external object not directly in contact with the user in addition to the user&#39;s head and may be located anywhere on the earpiece  12 . The sensors  18  may also contain an air microphone  42  and/or a bone conduction microphone  44  for detecting sound from the interior of the user&#39;s head and/or ambient sound, or one or more inertial sensors  46 ,  48  for detecting inertial changes or to further improve the accuracy of the location of the user&#39;s head. 
     A memory  33  operatively connected to the one or more processors  20  is also shown. The memory  33  may be used to store task profiles associated with performing one or more tasks. For example, a task profile may include proper head orientation and/or head movement associated with performing an activity. The information stored as a part of a task profile may be used to compare a user&#39;s current head orientation or position while performing a task with what is acceptable, likely to lead to improved performance, optimal, or otherwise desirable for performing the task. 
     One or more microphones  22  operatively connected to the earpiece  12  may be configured to receive audio inputs, where the audio inputs may originate from the user, a third-party, a machine, an animal, another earpiece, another electronic device, or even nature itself. The at least one microphone  22  need not be configured to receive every type of audio input. For example, one microphone may be configured to receive ambient environment audio inputs, and another microphone may be configured to receive audio inputs from the user such as speech. The gesture control interface  30  operatively connected to the earpiece  12  is configured to allow the user to control the earpiece  12 . The gesture control interface  30  may include at least one emitter  38  and at least one detector  40  to detect gestures from either the user, a third-party, an instrument, or a combination and transmit one or more gestures to one or more processors  20 . The gestures used with the gesture control interface  30  to control the earpiece  12  include, without limitation, touching, tapping, swiping, use of an instrument, or any combination of the gestures. Touching gestures used to control the earpiece  12  may be of any duration and may include the touching of areas not part of the gesture control interface  30 . Tapping gestures used to control the earpiece  12  may include any number of taps and need not be brief. Swiping gestures used to control the earpiece  12  may include a single swipe, a swipe changing direction at least once, a swipe with a time delay, a plurality of swipes, or any combination. The user interface may be used to identify a task which the user is performing or will perform, indicate when the task is complete, or indicate the user seeks feedback on their head or body position when performing the task. Of course, other types of user interfaces may be used including voice control interfaces. 
     The transceiver  32  disposed within the earpiece  12  may be configured to receive signals from and to transmit signals to a second earpiece of the user if the user is using more than one earpiece. The transceiver  32  may receive or transmit more than one signal simultaneously. The transceiver  32  may be of any number of types including a near field magnetic induction (NFMI) transceiver. The radio transceiver  34  disposed within the earpiece  12  is configured to receive signals from external electronic devices and to transmit those signals to the at least one processor  20 . The external electronic devices the radio transceiver  34  is configured to receive signals from may include Bluetooth devices, mobile devices, desktops, laptops, tablets, modems, routers, communications towers, cameras, watches, third-party earpieces, wearable devices, and other electronic devices capable of transmitting wireless signals. The radio transceiver  34  may receive or transmit more than one signal simultaneously. 
     The earpiece may communicate with one or more other body worn sensors including through the transceiver  32  or the radio transceiver  34 . Where other body worn sensors are used, the body worn sensors may provide additional kinematic data regarding body position. For example, a body worn sensor may be in the form of a watch or wristband and may include an inertial sensor. Similarly, body worn sensors may be integrated into clothing, footwear, jewelry, fashion accessories, or otherwise, to sense orientation or movement of the body including a neck of the user, a set of shoulders of the user, a torso of a user, a set of hips of the user, and a set of feet of the user. The measurements may then be communicated to the earpiece and used to determine a user&#39;s deviations from a task profile. 
     The LEDs  36  disposed within the earpiece  12  may be configured to emit light to convey to the user information concerning the earpiece  12 . The LEDs  36  may be in any area on the earpiece  12  suitable for viewing by the user or a third-party and may consist of as few as one diode. In addition, the LEDs  36  may be discernable by a human eye or an electronic device and need not have a minimum luminescence. 
     The battery  50  of the earpiece  12  is operatively connected to all the components within the earpiece  12 . The battery  50  may provide enough power to operate the earpiece  12  for a reasonable duration of time. The battery  50  may be of any type suitable for powering the earpiece  12 . However, the battery  50  need not be present in the earpiece  12 . Alternative power sources may be used to power the earpiece  12 . 
     One or more processors  20  are operatively connected to each component within the earpiece  12  and configured to receive measurements from one or more sensors  18  and use those measurements to determine a desired position of the user&#39;s head and variation with a task profile. The processors  20  may also be configured to receive signals from components other than the sensors  18 , such as one or more microphones  22 , the transceiver  32 , the radio transceiver  34 , or an external sensor not physically connected to the earpiece  12  to calculate the desired location for the user&#39;s head. It is also possible the user&#39;s head may not need to be moved after calculation by the one or more processors  20 . The processor(s)  20  are also configured to create a first audio signal containing instructions derived from the information conveyed by the sensors  18  to guide the user to the desired position of the user&#39;s head and to communicate the first audio signal to the speakers  16 . The first audio signal may also incorporate information from one or more microphones  22 , the transceiver  32 , or the radio transceiver  34  to improve the accuracy of the desired location for the user&#39;s head. In addition, one or more processors  20  may be configured to calculate a desired movement of the user&#39;s head from signals conveyed by one or more sensors  18  in order to create a second audio signal containing instructions derived from the information conveyed by the sensors  18  to guide the user as to the desired position of the user&#39;s head and to transmit the second audio signal to the speakers  16 . Like the first audio signal, the second audio signal may also incorporate information from one or more microphones  22 , the transceiver  32 , or the radio transceiver  34  to improve the accuracy of the desired movement of the user&#39;s head. In addition, one or more processors  20  may be configured to calculate the desired position of the user&#39;s head using signals concerning other parts of the user&#39;s body, clothing or other objects worn by the user, a surface the user is standing on, an external object not directly in contact with the user, or a combination of the aforementioned in addition to signals concerning the user&#39;s head conveyed by one or more sensors  18  or by one or more external sensors in order to create a third audio signal containing instructions derived from the information conveyed by the sensors  18  or one or more external sensors to guide the user to the desired location of the user&#39;s head and to transmit the third audio signal to the speakers  16 . The signals conveyed to the at least one processor  20  by the one or more sensors  18  or the one or more external sensors need not be in any order. Like the previous two signals, the third audio signal may also incorporate information from one or more microphones  22 , the transceiver  32 , or the radio transceiver  34  to improve the accuracy of the desired location for the user&#39;s head. 
     The guidance provided by the earpiece may be for the user to tilt their head up, to turn their head to the left or to the right, to tilt their head down, to lift their head up, to lower their head, to tilt their head to one side or the other, or combinations of these or other head movements which will guide the head from a current position to a desired position. Therefore, the processor may convert specific measurements such as degrees into more understandable language such as “Turn your head slightly towards the left”, “Eyes forward”, “Look up”, “Slowly move your head down and towards the right.” “Move your head to the left until I say stop. Stop.” It is to also be understood the audio guidance need not be verbal. For example, a sound can be played which changes pitch or volume until the user moves their head to the proper position. 
       FIG. 4  illustrates a flowchart of one example of a method  100  of the invention. At least one sensor determines  102  a current orientation of a user&#39;s head from at least one movement reading of the user&#39;s head by the at least one sensor. The user&#39;s head may be stationary or in motion, and the at least one movement reading of the user&#39;s head may be either two-dimensional or three-dimensional and may include one or more data points. At least one processor then calculates  104  a desired position of the user&#39;s head based on the at least one movement reading of the user&#39;s head. The user&#39;s head does not necessarily have to be moved from its current position and the calculation  104  may incorporate data in addition to the at least one movement reading to ascertain the desired position of the user&#39;s head. The at least one processor then creates  106  a first audio signal which contains instructions derived from the at least one movement reading of the user&#39;s head to guide the user to the desired position of the user&#39;s head. The first audio signal may be of any intensity not causing undue harm to the user&#39;s tympanic membranes and the instructions may be verbal instructions, sounds associated with specific instructions known or knowable by the user, sounds which coax the head of the user to its desired position without conscious input by the user, or a combination. The at least one processor then communicates  108  the first audio signal to at least one speaker. The first audio signal may be disaggregated into multiple signals and communicated  108  to more than one speaker if multiple instructions are involved. The at least one speaker then transduces  110  the first audio signal to at least one of a user&#39;s ears. The first audio signal may be continuous or discontinuous. 
     It is also contemplated a record of the user&#39;s head positions, head movement, and guidance or corrections provided may be stored in either the earpieces or at a connected device such as a mobile device. This may be further processed to provide additional insight regarding a user&#39;s performance in a task. It may also be used to predict a user&#39;s head movement in response to a stimulus, so the guidance process may begin earlier or even pre-emptively. 
     Where both a left earpiece and a right earpiece are present, position, orientation and movement of a human head may be tracked in a variety of ways including through monitoring pitch, roll, and yaw. Pitch may be considered movement of the head forward or back. Roll may be considered movement tilting the head from side to-side, and yaw may be considered turning left and right. The inertial sensor may include a 3-axis accelerometer, a 3-axis gyroscope, an a 3-axis magnetometer. Having an inertial sensor in both the right and left earpiece allows for greater accuracy then in one ear piece. 
     Various tasks may have preferred position or movement profiles associated with them as a task profile. These profiles may be determined in various ways including from simply observing sensed data from one or more experts performing the tasks in what they believe to be the correct manner. A profile may also be created from a user performing a task in a particularly successful manner or may be constructed individually for a user. In operation, a comparison is made between the position or movement of the user&#39;s head and one or more positions or movements associated with the task profile. If the difference in position exceeds a preset criteria or threshold, then the user is guided by the earpieces to adjust their head position to the desired head position. Similarly, if the earpiece is in communication with other body worn sensors than the profile may contain position or movement for other parts of the body. In operation, a comparison is made between the position or movement of the user&#39;s body and one or more positions or movements associated with the task profile. If the difference in position exceeds a preset criteria or threshold, then the user is guided by the earpieces to adjust their body position to the desired body position such as through audio feedback from the earpiece. 
     Therefore, various methods, systems, and apparatus have been shown and described for allowing for one or more earpieces to provide feedback to a user regarding their head and/or body placement. Although various specific examples have been shown and described, it is to be understood the present invention contemplates variations in the structure and components of the earpieces and the methods in which user head or body position is determined and, in the way, feedback is provided.