Patent Description:
<CIT> discloses a personal hygiene device capable of providing the user with feedback which can assist the user in his/her personal hygiene routines.

Various forms of tracking the location of an oral care device within a user's mouth have been attempted. Tracking the location of the oral care device allows for effective feedback to a user with respect to the user's oral hygiene practices. For example, if the location of a brush head is tracked within the user's mouth, portions of teeth not yet cleaned may be identified so that the user can focus on those areas.

However, known attempts to track the location of a brush within the user's mouth have disadvantages. For example, video-based tracking devices are designed to track the handle of a brush in relation to the tracked face. The localization of the brush tip inside the mouth is then based on extrapolation, which can be a cause of inaccuracy. Moreover, occlusion by the device operating hand challenges the robustness of this method.

As another example, locally generated static magnetic fields have been used to determine position and orientation in relation to a user's head. The user needs to wear ear plugs that sense of the appliance position and orientation in relation to the ear plugs. Inserting earbuds for appliance tracking is acceptable for infrequent tasks, but is likely unaccepted every time you brush your teeth.

As a further example, accelerometers and/or gyroscopes can determine orientation of the toothbrush in relation to gravity field, and relative orientation on other axes. Accelerometers and gyroscopes, however, cannot provide exact information on where the user is brushing in the mouth, since the reference to the teeth is missing.

Accordingly, there is a need in the art for improved systems and methods for tracking the location of an oral care device within the mouth of a user.

The present disclosure is directed to inventive methods and systems for accurately tracking the location of an oral care device within the mouth of a user. Various embodiments and implementations herein are directed to a system that includes an oral care device having one or more light sources configured to emit light from a portion of the oral care device. A computing device, including a processor and a non-transitory storage medium for storing program code, is programmed to track a user's face and the location of the oral care device according to the light emitted by the light source. The computing device is further configured to estimate the location of the user's teeth, and to compare the location of the light source with the estimated location of the oral care device, to determine which teeth are currently being cleaned. The system may also be configured to determine the orientation of the oral care device within the mouth of the user, to determine which side of the teeth are currently being cleaned. The system may include a dedicated application for showing a user which teeth have been cleaned and which teeth may still require cleaning.

In various implementations herein, a processor or controller may be associated with one or more storage media (generically referred to herein as "memory," e.g., volatile and non-volatile computer memory such as RAM, PROM, EPROM, and EEPROM, floppy disks, compact disks, optical disks, magnetic tape, etc.). In some implementations, the storage media may be encoded with one or more programs that, when executed on one or more processors and/or controllers, perform at least some of the functions discussed herein. Various storage media may be fixed within a processor or controller or may be transportable, such that the one or more programs stored thereon can be loaded into a processor or controller so as to implement various aspects of the present invention discussed herein. The terms "program" or "computer program" are used herein in a generic sense to refer to any type of computer code (e.g., software or microcode) that can be employed to program one or more processors or controllers. In addition, the "program" or "computer code" is to be understood as being stored on a non-transitory, computer readable medium.

The present disclosure is directed to inventive methods and systems for accurately tracking the location of an oral care device within the mouth of a user. More generally, Applicant has recognized and appreciated that it would be beneficial to track the location of oral care device by monitoring a light transmitting through the cheek of a user. Various embodiments and implementations herein are directed to a system that includes an oral care device having one or more light sources configured to emit light from a portion of the oral care device. A computing device, including a processor and a non-transitory storage medium for storing program code, is programmed to track a user's face and the location of the oral care device according to the light emitted by the light source. The computing device is further configured to estimate the location of the user's teeth, and to compare the location of the light source with the estimated location of the oral care device, to determine which teeth are currently being cleaned. The system may also be configured to determine the orientation of the oral care device within the mouth of the user, to determine which side of the teeth are currently being cleaned. The inventive system may include a dedicated application for showing a user which teeth have been cleaned and which teeth may still require cleaning.

Referring to <FIG>, in one embodiment, is a representation of a system <NUM> for tracking an oral care device within the mouth of a user. System <NUM> comprises an oral care device <NUM> such as a toothbrush, oral irrigator, or other device, having at least one light source <NUM> located near or on the tip of oral care device <NUM>. Note that the light source <NUM> only needs to be located at position of the oral care device such that it will enter a user's oral cavity during use. The light source <NUM> may be configured to emit visible light or it may emit light outside of the visible spectrum, such as infrared light. In an embodiment, light source <NUM> is located on the side of the oral care device opposite the operable side. For example, if the oral care device is a toothbrush, the light source may be located on the side opposite the bristles. To avoid putting an expensive light source in a disposable brush head (or the equivalent on a different oral care device), the light source may be placed in the handle of the oral care device, and each replaceable head may have optical fiber for conducting the light from within the handle to a point where it may be emitted from the head of oral care device <NUM>. In an alternate embodiment, the light may be also be emitted from the operable side of oral care device <NUM>, such as from the bristles of a toothbrush.

In an embodiment, the oral care device <NUM> has at least one other light source <NUM>, configured to emit light in a different direction (radially from oral care device <NUM>) from light source <NUM>. For example, light source <NUM> may be placed on the side of oral care device <NUM>, such that it emits light at approximately a <NUM> degree angle with respect to the light emitted by light source <NUM>. Light source <NUM> and light source <NUM> may further emit light of different frequencies (i.e. light source <NUM> may emit light with a first frequency while light source <NUM> emits light with a second frequency), or modulated with different codes (i.e. light source <NUM> may transmit light embedded with a first code, while light source <NUM> may transmit light embedded with a second code). In an embodiment, oral care device <NUM> may include a microchip or other computing device sufficient for modulating the light emitted by light source <NUM> and light source <NUM> with differing codes, or to cause light source <NUM> and light source <NUM> to emit light of different frequencies. Furthermore, oral care device <NUM> includes a sensor <NUM>, such as an accelerometer or a gyroscope.

System <NUM> further comprises a computing device <NUM>, which may include a processor and a non-transitory storage medium, containing program code to cause the processor to execute an algorithm in accordance with the various embodiments described herein. Computing device <NUM> may be a general purpose computer, a mobile device, a custom dedicated computing device, or any other computing device capable of storing and executing the program described herein. Computing device <NUM> may be configured with a face tracking algorithm, as are known in the art. Computing device <NUM> is operable connected to at least one optical sensor <NUM>, such as a camera, that is capable of detecting the light emitted by light source <NUM>, and light reflected from the face of a user of sufficient resolution that a facial tracker may be able to locate and track a face. Optical sensor <NUM> may be a standard CMOS camera, or may be a camera embedded on a mobile device. Optical sensor <NUM> may also comprise a plurality of optical sensors. For example, one optical sensor may be configured to receive infrared light emitted by light source <NUM> while another is configured to view a user's face.

Where computing device <NUM> is a mobile device, the program operated by computing device <NUM> may be a mobile application, which may also be configured to display a graphical interface <NUM> of a model of a user's teeth. Where computing device <NUM> is a device apart from a mobile device, computing device <NUM> may cooperate with a mobile application or with another application, for displaying a model of the user's teeth. Alternatively, computing device <NUM> may be associated with a display (apart from a mobile device) for displaying the model of the user's teeth.

Computing device <NUM> and optical sensor <NUM> may be associated with a mirror, such as a "smart mirror" where a user could view their face while using oral care device <NUM>. For example, a smart mirror may comprise a display behind a half-translucent mirror.

Referring to <FIG>, there is shown a flow chart of a method <NUM> for tracking the location of an oral care device within the oral cavity of a user. The method utilizes one or more embodiments of the systems described or otherwise envisioned herein. For example, method <NUM> may use system <NUM> described above, including oral care device <NUM> and computing device <NUM>.

At step <NUM>, a signal from optical sensor <NUM> is received by computing device <NUM>. This signal may be received over a hard connection, or via a wireless connection.

At step <NUM>, the location of light source <NUM> is determined from the received emitted light. As light source <NUM> is located near the tip of oral care device <NUM>, the light source will project the light from within the user's mouth, and through the user's skin, when in use. Thus, the optical sensor <NUM> is configured to receive light emitted by light source <NUM> as it is transmitted from within the user's mouth and through the skin of the user's cheek. This step may include identifying a frequency or embedded code within the light so as to determine the received light is actually from light source <NUM> and not some other source.

At step <NUM>, computing device <NUM> may process the signals received from optical sensor <NUM> so as to identify at least one feature of a user's face. This step may be accomplished by facial identification or tracking software as is known in the art. For example, the facial tracking software may identify a feature of the user's face, such as an outline of the user's face, the location of the user's eyes, lips, etc. At step <NUM>, computing device <NUM> may process the signals received by optical sensor <NUM> so as to identify the location of light source <NUM> from the light emitted by light source <NUM>.

At step <NUM>, using the facial tracking software, the location of the user's oral cavity or teeth is estimated, relative to the identified facial feature. For example, computing device <NUM>, using the facial tracking software, may identify at least one facial feature and use this feature to track the user's face. Based on known relations between facial features, such as lips, eyes, etc. the location of the user's teeth may be estimated.

At step <NUM>, the location of the light source is compared with the estimated location of the teeth to determine the position of oral care device <NUM> with respect to the user's teeth. At step <NUM>, from this determined position, the teeth being cleaned by the oral care device may also be determined (i.e. the oral care device is assumed to be cleaning the teeth it is closest to).

As shown in <FIG>, at step <NUM>, a measurement may be taken to determine the orientation of oral care device <NUM> within the user's mouth. For example, the intensity of the light emitted by light source <NUM> may be measured relative to a predetermined level or relative to a previously measured level, to determine whether light source <NUM> is pointing towards or away from optical sensor <NUM>. For example, if the received light is diminished it may be assumed that oral care device is oriented away from optical sensor <NUM>.

Alternately, computing device <NUM> may distinguish between the light received from two separate light sources <NUM>, <NUM> pointing in different directions. For example, if the light source <NUM> is positioned on the back face of oral care device <NUM> and light source <NUM> is positioned on a side of oral care device <NUM>, if the light emitted by light source <NUM> is stronger than the light emitted by light source <NUM>, it may be assumed that the back of oral care device is facing away from optical sensor <NUM> while light source <NUM>, i.e. the side of oral care device <NUM>, is facing toward optical sensor. If the orientation, with respect to optical sensor <NUM> is known it may be assumed that a particular side of the teeth is being cleaned.

To distinguish between the lights received by light sources <NUM> and <NUM>, (note that in alternate embodiments, more than two light sources may be used), and as discussed above, light source <NUM> may have one frequency or be embedded with one code, while light source <NUM> may have another frequency or be embedded with another code. Accordingly computing device <NUM> may differentiate between the light signals received by optical sensor <NUM>, in order to determine the orientation of oral care device <NUM>. In addition to light source <NUM>, other sensors, such as an accelerometer or gyroscope may be used to determine the orientation of the brush within the user's mouth. Furthermore, another light source may be placed on the handle of the oral care device, which may be compared with the light received from within the user's cheek, to further determine the position or orientation of oral care device <NUM>.

In an alternate embodiment, computing device <NUM> may determine the orientation or position of oral care device by measuring size of a pattern projected onto the interior of the cheek of a user by at least one light source (such as light source <NUM>, although it may be any light-projecting element) and detected by optical sensor <NUM> through the cheek of the user. For example, two light sources arranged to emit light at a predetermined angle, may be used to project the pattern onto the cheek of the user. As shown in <FIG>, if oral care device <NUM> is far from the interior of the cheek of a user, the size of the pattern projected onto the cheek may be relatively large. However, as shown in <FIG>, as oral care device <NUM> is moved closer to the cheek of a user, the pattern may shrink. Computing device <NUM> may thus determine the position and orientation of oral care device <NUM> by measuring the size of the pattern relative to the face of the user, or relative to some other metric. For example, computing device <NUM> may receive from optical sensor <NUM> both the image of the user's face and the size of the pattern shining through the user's cheek. By comparing the size of the pattern to the size of the user's cheek, computing device <NUM> may determine that oral care device is positioned on the far side of the user's mouth (relative to the cheek being projected onto), and is oriented such that oral cleaning device is cleaning the interior of the user's teeth, as shown in <FIG>. Alternately, if the pattern is very small with respect to the user's cheek, computing device may determine that oral care device <NUM> is positioned on the near side of the user's mouth relative to the cheek being projected onto, and is cleaning the outside of the user's teeth. The pattern of light may be any detectable predetermined, random, or pseudorandom pattern. The pattern may be emitted by several light sources, or by a single light source.

Referring back to <FIG>, at step <NUM>, according to the determined orientation of oral care device <NUM>, the side of the teeth currently being cleaned may be identified. For example, if it is determined that oral care device <NUM> is oriented such that the back is facing the optical sensor <NUM>, it may be determined the front of the user's teeth is currently being cleaned.

On occasion, light from each of the light sources <NUM>, <NUM>, etc., may completely disappear, or be severely diminished. When this occurs, it may be assumed that the tip of the oral care device is behind the user's teeth. To continue tracking the location of the tip, the position of the tip may be extrapolated from its last known point, or, to generate a history of the brush has been, a path traveled may be interpolated (at optional step <NUM>) when the tip reappears from behind the teeth and light is once again received. By using the point that the tip was last viewed, and the point where it reappears, the path traveled by the brush may be estimated.

Tracking the location of oral care device <NUM> may be used in conjunction with an application, such as a mobile application, or on a display of general purpose computer or a smart mirror. The application may be used for informing a user which teeth have been cleaned, and which teeth still require cleaning. For example, the application may be display an abstract or 3D model teeth: clean teeth may be displayed as white, or having some other marker on them, while the teeth yet to be cleaned may be displayed as covered with virtual plaque, or have some other marker to identify them. The application may provide feedback on brushing time per location within the mouth, as it corresponds to individual teeth or teeth sections, as an indication of brush quality.

Alternatively, feedback may be given via a brush indicator light (which may or may not be the same as light source <NUM>) which may indicate whether the user has spent enough time brushing a particular location (i.e. an amber light may indicate that not enough time has been spent at a location, while a green light may be indicate that the location is sufficiently cleaned). Indeed, any information, such as the movement speed of toothbrush <NUM>, the force applied to toothbrush <NUM>, etc. may be communicated by an indicator light by changing the frequency/color of the light. For example, computing device <NUM> may determine that the user is brushing too quickly, i.e. the position of toothbrush <NUM> is changing at a rate greater than a predetermined value. This may then be communicated back to toothbrush <NUM> via a microchip or other computing device located on toothbrush <NUM>, and the color of the indicator light may be changed accordingly. Thus, if the user is brushing too quickly, indicator light may turn yellow, but may turn blue if the user is brushing at the proper speed. Alternately, a sensor <NUM> located on toothbrush <NUM> may calculate that too much force is being applied to toothbrush <NUM>. The microchip located on toothbrush <NUM>, or computing device <NUM>, or a combination of computing devices acting in concert (including a microchip or computing device <NUM>), may then instruct the indicator light to change colors to notify the user that too much force is being applied. It will be appreciated that any value, such as time spent at a position, force of brushing, or rate of brushing, may be calculated by computing device <NUM> or by a chip located on toothbrush <NUM> and communicated to the user by changing the color/frequency of one or more indicator lights located on toothbrush <NUM>. In yet another embodiment, the microchip located on toothbrush <NUM> may communicate with computing device <NUM> via code embedded in the light emitted by the indicator light. For example, if a sensor <NUM> is reading that too much force is being applied, this may be communicated to computing device <NUM> by modulating the emitted light with a code that may be received by optical sensor <NUM> and interpreted by computing device <NUM>. Computing device <NUM> may then notify user that too much force is being applied via an app on a mobile device or through a screen located on toothbrush <NUM> or by some other indicator. In this way any value detected by toothbrush <NUM>, such as force applied, rate of brushing, etc., may be communicated to computing device <NUM>. The value communicated to computing device <NUM> may be used to notify a user in real time whether the user is brushing properly, or may be stored and delivered to the user as a summary.

Optionally, a user or dentist may be able to enter where problem areas are that require specific brushing attention, which may be reflected in the amount of time that the application recommends a user spend on certain portion of teeth, or by otherwise notifying of their need to focus on that spot. The application may further store brushing behavior, as determined by the location of the oral care device, and bring it to the attention of a user or a dentist.

In addition, as computing device <NUM> tracks oral care device <NUM>, computing device <NUM> may begin to form a model of the user's teeth. For example, if the user is missing molars, or wisdom teeth, computing device <NUM> recognize that oral care device <NUM> does not travel to these points in the mouth and may accordingly update or form a model of the user's teeth.

Only terms clearly indicated to the contrary will refer to the inclusion of exactly one element of a number or list of elements. In general, the term "or" as used herein shall only be interpreted as indicating exclusive alternatives (i.e. "one or the other but not both") when preceded by terms of exclusivity, such as "one of.

In the claims, as well as in the specification above, all transitional phrases such as "comprising," "including," "having," "containing," and the like are to be understood to be openended, i.e., to mean including but not limited to.

Claim 1:
A system for monitoring the location of an oral care device (<NUM>) within the mouth of a user, the system comprising:
an oral care device (<NUM>) having a light source (<NUM>, <NUM>);
an optical sensor (<NUM>) configured to receive light emitted by the light source (<NUM>) and to receive light reflected from a user using the oral care device (<NUM>);
a computing device (<NUM>) adapted to receive and process signals generated by the optical sensor (<NUM>), and programmed to:
detect facial features of the user from the received reflected light;
estimate the location of the user's teeth according to the detected facial features;
determine a location of the light source (<NUM>, <NUM>) according to the received emitted light;
compare the location of the light source (<NUM>, <NUM>) to the location of the teeth to estimate the position of the oral care device (<NUM>) with respect to the teeth of the user.