Smart mirror system and methods of use thereof

A system comprises a frame, a mirror coupled to the frame, an electronic display device coupled to the frame, a camera coupled to the frame, one or more processors, and one or more memory devices. The one or more processors are configured to execute instructions that are stored in the one or more memory devices. The one or more processors executing the instructions causes the system to continuously display, on the electronic display device, a real-time video feed of a user within a field of view of the camera; identify and object within the field of view of the camera; and, subsequent to the identification, modify the continuously displayed real-time video feed based on at least the identified object.

TECHNICAL FIELD

The present disclosure relates to an interactive mirror device, more specifically, the present disclosure relates to an interactive mirror with the ability to recognizes faces and objects.

BACKGROUND

In their daily lives, people perform many tasks in front of mirrors, such as brushing their teeth, shaving, applying makeup, and generally getting ready for their day. These activities are often undertaken in poor lighting conditions and can cause people problems in not sufficiently being prepared for their day. For example, a person may apply their makeup incorrectly due to inadequate lighting in their bathroom or bedroom where their mirror is located. In another example, a person may not be able to see their face properly when shaving, causing them to miss spots. In an additional example, the ambient lighting conditions may not be a realistic estimate of the actual lighting conditions a person will experience throughout the day, and thus the person cannot ensure that they have sufficiently prepared themselves when they can only view themselves in in adequate lighting. The present disclosure is directed to solving these problems and addressing other needs.

SUMMARY

According to implementations of the present disclosure, a system comprises a frame, a mirror, an electronic display device, a camera, one or more processors, and one or more memory devices. The mirror is coupled to the frame. The electronic display device is coupled to the frame such that the electronic display device is positioned adjacent to at least a portion of the mirror. The camera is coupled to the frame. The one or more processors are configured to execute instructions stored in the one or more memory devices such that the system is caused to: continuously display, on the electronic display device, a real-time video feed of at least a portion of a user positioned within a field of view of the camera; identify an object within the field of view of the camera; and subsequent to the identification, modify the continuously displayed real-time video feed based on at least the identified object.

According to implementations of the present disclosure, a system comprises a frame, a mirror, an electronic display device, a camera, one or more processors, and one or more memory devices. The mirror is coupled to the frame. The electronic display device is coupled to the frame such that the electronic display device is positioned adjacent to at least a portion of the mirror. The camera is coupled to the frame. The one or more processors are configured to execute instructions stored in the one or more memory devices such that the system is caused to: continuously display, on the electronic display device, a real-time video feed of at least a portion of a user positioned within a field of view of the camera; and subsequent to receiving a prompt, modify the continuously displayed real-time video feed based on at least the prompt.

According to implementations of the present disclosure, a system comprises a frame, a mirror, an electronic display device, a camera, one or more processors, and one or more memory devices. The mirror is coupled to the frame. The electronic display device is coupled to the frame such that the electronic display device is positioned adjacent to at least a portion of the mirror. The camera is coupled to the frame. The one or more processors are configured to execute instructions stored in the one or more memory devices such that the system is caused to: monitor, using the camera, a user positioned within a field of view of the camera; and continuously display, on the electronic display device, a real-time video feed of a first portion of the user, such that a second portion of the user is visible to the user in the mirror; receive a prompt; and subsequent to the receiving of the prompt, modify the displayed first portion of the user in the continuously displayed real-time video feed.

According to implementations of the present disclosure, a system comprises a frame, a mirror, an electronic display device, a camera, one or more processors, and one or more memory devices. The mirror is coupled to the frame. The electronic display device is coupled to the frame such that the electronic display device is positioned adjacent to at least a portion of the mirror. The camera is coupled to the frame. The one or more processors are configured to execute instructions stored in the one or more memory devices such that the system is caused to: monitor, using the camera, a user positioned within the field of view of the camera; continuously display, on the electronic display device, a real-time video feed of at least a portion of the user; determine a mood of the user; and subsequent to the determination of the mood of the user, modify the continuously displayed real-time video feed based on at least the determined mood of the user.

The foregoing and additional aspects and implementations of the present disclosure will be apparent to those of ordinary skill in the art in view of the detailed description of various embodiments and/or implementations, which is made with reference to the drawings, a brief description of which is provided next.

DETAILED DESCRIPTION

Referring toFIG.1, a smart mirror system10according to the present disclosure includes a mirror12, one or more electronic displays14, one or more sensors16, one or more light sources18, and one or more cameras20. The system10generally also includes at least one processor22with a memory24. The memory24generally contains processor-executable instructions that when executed by the processor22, run an operating system and/or an application on the display14. The mirror12is of a type that is generally referred to as a one-way mirror, although it is also sometimes referred to as a two-way mirror. The mirror12is configured transmit a first portion of light that is incident on its surfaces to the other side of the mirror12, and to reflect a second portion of the light that is incident on its surfaces. This may be accomplished by applying a thin layer of a partially reflective coating to a generally transparent substrate material, such that less than all of the incident light is reflected by the partially reflecting coating. The remaining light is transmitted through the mirror12to the other side. Similarly, some light that strikes the mirror12on a side opposite the side where a user is standing will be transmitted through the mirror12, allowing the user to see that transmitted light. This partially reflective coating can generally be applied to surface of the substrate material on the display-side of the substrate material, the user-side of the substrate material, or both. Thus, the partially reflective coating can be present on the surface of one or both of the display-side and the user-side of the mirror12. In some implementations, the partially reflective coating is made of silver. The generally transparent material can be glass, acrylic, or any other suitable material. The mirror12can have a rectangular shape, an oval shape, a circle shape, a square shape, a triangle shape, or any other suitable shape. The processor22is communicatively coupled with the electronic display14, the one or more sensors16, the one or more light sources18, and the one or more cameras20.

Referring now toFIG.2, the mirror12can be mounted on a base26. The mirror12could also be directly mounted in a counter, a wall, or any other structure. The electronic display14is mounted on, coupled to, or otherwise disposed on a first side of the mirror12, while a sensor frame28containing the one or more sensors16is disposed at an opposing second side of the mirror12. The side of the mirror12where the display14is located is generally referred to as the display-side of the mirror12. The side of the mirror12where the sensor frame28is located is generally referred to as the user-side of the mirror12, as this is the side of the mirror12where the user will be located during operation.

The electronic display14is generally mounted in close proximity to the surface of the display-side of the mirror12. The electronic display14can be any suitable device, such as an LCD screen, an LED screen, a plasma display, an OLED display, a CRT display, an LED dot matrix display, or the like. In some implementations, the LED dot matrix display is a relatively low resolution LED display. For example, the relatively low resolution LED display can include between about 1 and about 5 LEDs per square inch, between about 1 and about 10 LEDs per square inch, between about 1 and about 25 LEDs per square inch, or between about 1 and about 50 LEDs per square inch. Due to the partially reflective nature of the mirror12, when the display14is activated (e.g. turned on and emitting light to display an image), a user standing on the user-side of the mirror12is able to view any portion of the display14that is emitting light through the mirror12. When the display14is turned off, light that is incident on the user-side of the mirror12from the surroundings will be partially reflected and partially transmitted. Because the display14is off, there is no light being transmitted through the mirror12to the user-side of the mirror12from the display-side. Thus, the user standing in front of the mirror12will see their reflection due to light that is incident on the user-side of the mirror12and is reflected off of the mirror12back at the user. When the display14is activated, a portion of the light produced by the display14that is incident on the mirror12from the display-side is transmitted through the mirror12to the user-side. The mirror12and the display14are generally configured such that the intensity of the light that is transmitted through the mirror12from the display14at any given point is greater than the intensity of any light that is reflected off of that point of the mirror12from the user-side. Thus, a user viewing the mirror12will be able to view the portions of the display14that are emitting light, but will not see their reflection in the portions of those mirror12through which the display light is being transmitted.

The electronic display14can also be used to illuminate the user or other objects that are located on the user-side of the mirror12. The processor22can activate a segment of the display14that generally aligns with the location of the object relative to the mirror12. In an implementation, this segment of the display14is activated responsive to one of the one or more sensors16detecting the object and its location on the user-side of the mirror12. The segment of the display14can have a ring-shaped configuration which includes an activated segment of the display14surrounding a non-activated segment of the display14. The non-activated segment of the display14could be configured such that no light is emitted, or could be configured such that some light is emitted by the display14in the non-activated segment, but it is too weak or too low in intensity to be seen by the user through the mirror12. In an implementation, the activated segment of the display14generally aligns with an outer periphery of the object, while the non-activated segment of the display14generally aligns with the object itself. Thus, when the object is a user's face, the user will be able to view the activated segment of the display14as a ring of light surrounding their face. The non-activated segment of the display14will align with the user's face, such that the user will be able to see the reflection of their face within the ring of light transmitted through the mirror12. In another implementation, the non-activated segment of the display14aligns with the object, and the entire remainder of the display14is the activated segment. In this implementation, the entire display14is activated except for the segment of the display14that aligns with the object.

Generally, the system10includes one or more sensors16disposed in the sensor frame28. The sensor frame28is mounted on, couple to, or otherwise disposed at the second side (user-side) of the mirror12. The sensors16are generally located within a range of less than about five inches from the user-side surface of the mirror12. In other implementations, the sensors16could be disposed between further away from the surface of the mirror12, such as about between about 5 inches and about 10 inches. The sensors16are configured to detect the presence of a hand, finger, face, or other body part of the user when the user is within a threshold distance from the mirror12. This threshold distance is the distance that the sensors16are located away from the user-side surface of the mirror12. The sensors16are communicatively coupled to the processor22and/or memory24. When the sensors16detect the presence of the user aligned with a certain point of the mirror12(and thus the display14), the processor22is configured to cause the display14to react as if the user had touched or clicked the display14at a location on the display14corresponding to the point of the mirror12. Thus, the sensors16are able to transform the mirror12/display14combination into a touch-sensitive display, where the user can interact with and manipulate applications executing on the display14by touching the mirror12, or even bringing their fingers, hands, face, or other body part in close proximity to the user-side surface of the mirror12. In some implementations, the sensors16can include a microphone that records the user's voice. The data from the microphone can be sent to the processor22to allow the user to interact with the system10using their voice.

The one or more sensors16are generally infrared sensors, although sensors utilizing electromagnetic radiation in other portions of the electromagnetic spectrum could also be utilized. The sensor frame28can have a rectangular shape, an oval shape, a circular shape, a square shape, a triangle shape, or any other suitable shape. In an implementation, the shape of the sensor frame28is selected to match the shape of the mirror12. For example, both the mirror12and the sensor frame28can have rectangular shapes. In another implementation, the sensor frame28and the mirror12have different shapes. In an implementation, the sensor frame28is approximately the same size as the mirror12and generally is aligned with a periphery of the mirror12. In another implementation, the sensor frame28is smaller than the mirror12, and is generally aligned with an area of the mirror12located interior to the periphery of the mirror12. In a further implementation, the sensor frame28could be larger than the mirror12.

In an implementation, the mirror12generally has a first axis and a second axis. The one or more sensors16are configured to detect a first axial position of an object interacting with the sensors16relative to the first axis of the mirror12, and a second axial position of the object interacting with the sensors16relative to the second axis of the mirror12. In an implementation, the first axis is a vertical axis and the second axis is a horizontal axis. Thus, in viewing the sensor frame28from the perspective of the user, the sensor frame28may have a first vertical portion28A and an opposing second vertical portion28B, and a first horizontal portion28C and an opposing second horizontal portion28D. The first vertical portion28A has one or more infrared transmitters disposed therein, and the second vertical portion28B has one or more corresponding infrared receivers disposed therein. Each individual transmitter emits a beam of infrared light that is received by its corresponding individual receiver. When the user places a finger in close proximity to the mirror12, the user's finger can interrupt this beam of infrared light such that the receiver does not detect the beam of infrared light. This tells the processor22that the user has placed a finger somewhere in between that transmitter/receiver pair. In an implementation, a plurality of transmitters is disposed intermittently along the length of the first vertical portion28A, while a corresponding plurality of receivers is disposed intermittently along the length of the second vertical portion28B. Depending on which transmitter/receiver pairs detect the presence of the user's finger (or other body part), the processor22can determine the vertical position of the user's finger relative to the display14. The first axis and second axis of the mirror12could be for a rectangular-shaped mirror, a square-shaped mirror, an oval-shaped mirror, a circle-shaped mirror, a triangular-shaped mirror, or any other shape of mirror.

The sensor frame28similarly has one or more infrared transmitters disposed intermittently along the length of the first horizontal portion28C, and a corresponding number of infrared receivers disposed intermittently along the length of the second horizontal portion28D. These transmitter/receiver pairs act in a similar fashion as to the ones disposed along the vertical portions28A,28B of the sensor frame28, and are used to detect the presence of the user's finger and the horizontal location of the user's finger relative to the display14. The one or more sensors16thus form a two-dimensional grid parallel with the user-side surface of the mirror12with which the user can interact, and where the system10can detect such interaction.

In other implementations, the sensor frame28may include one or more proximity sensors, which can be, for example, time of flight sensors. Time of flight sensors do not rely on separate transmitters and receivers, but instead measure how long it takes an emitted signal to reflect off on an object back to its source. A plurality of proximity sensors on one edge of the sensor frame28can thus be used to determine both the vertical and horizontal positions of an object, such as the user's hand, finger, face, etc. For example, a column of proximity sensors on either the left or right edge can determine the vertical position of the object by determining which proximity sensor was activated, and can determine the horizontal position by using that proximity sensor to measure how far away the object is from the proximity sensor. Similarly, a row of proximity sensors on either the top or bottom edge can determine the horizontal position of the object by determining which proximity sensor was activated, and can determine the vertical position by using that proximity sensor to measure how far away the object is from the proximity sensor. When one or more of the proximity sensors detect the presence of the user's finger or other body part (e.g., hand, face, arm, etc.), the user's finger or other body part is said to be in the “line of sight” of the sensor.

The sensors16in the sensor frame28(whether IR transmitter/receiver pairs and/or proximity sensors) can be used by the system10to determine different types of interactions between the user and the system10. For example, the system10can determine whether the using is swiping horizontally (left/right), vertically (up/down), diagonally (a combination of left/right and up/down), or any combination thereof. The system10can also detect when the user simply taps somewhere instead of swiping. In some implementations, the sensor frame28is configured to detect interactions between the user and the system10when the user is between about 3 centimeters and about 15 centimeters from the surface of the mirror12. A variety of different applications and programs can be run by the processor22, including touch-based applications designed for use with touch screens, such as mobile phone applications. Generally, any instructions or code in the mobile phone application that rely on or detect physical contact between a user's finger (or other body part) and the touch-sensitive display of the mobile phone (or other device) can be translated into instructions or code that rely on or detect the user's gestures in front of the mirror12.

Any applications that are run by the processor22and are displayed by the display14can be manipulated by the user using the sensors16in the sensor frame28. Generally, the sensors16in the sensor frame28detect actions by the user, which causes the processor22to take some action. For example, a user-selectable icon may be displayed on the display14. The user can select the user-selectable icon, which triggers the processor22to take some corresponding action, such as displaying a new image or screen on the display14. The triggering of the processor22due to the user's interaction with the system10can be affected using at least two different implementations.

In the first implementation, the processor22is triggered to take some action once the user's finger (or other body part) is removed from the proximity of the display14and/or the sensors16, e.g., is removed from the line of sight of the sensors16. For example, when the user wishes to select a user-selectable icon displayed on the display14, they can move their finger into close proximity with the displayed user-selectable icon without touching the mirror12, such that the user's finger is in the line of sight of the sensors16. However, coming into the line of sight of the sensors16does not trigger the processor22to take any action, e.g., the user has not yet selected the icon by placing their finger near the icon on the display14. Once the user removes their finger from the proximity of the icon on the display14and the line of sight of the sensor16, the processor22is triggered to take some action. Thus, the user only selects the user-selectable icon once the user removes their finger from the proximity of the icon.

In the second implementation, the processor22is triggered to take some action when the user's finger (or other body part) is moved into proximity of the display14and/or the sensors16, e.g., is moved into the line of sight of the sensors16. In this implementation, as soon as the user moves their finger into close proximity of a user-selectable icon on the display14(e.g., moves their finger into the line of sight of the sensors16), the processor22is triggered to take some action corresponding to the section of the icon. Thus, the user does not have to move their finger near the icon/sensors16and then remove their finger in order to select the icon, but instead only needs to move their finger near the icon/sensors16. In some such implementations, selection of an icon requires that the user hold its finger near the icon for a predetermined amount of time (e.g., 1 second, 1.5 seconds, 2 seconds, 3 seconds, etc.).

Other methods of controlling the system10can also be used. For example, the system10can include a microphone (or communicate with a device that includes a microphone) to allow for voice control of the system10. The system10can also include one or more physical buttons or controllers the user can physically actuate in order to interact with and control the system10. In another example, the system10could communicate with a separate device that the user interacts with (such as a mobile telephone, tablet computer, laptop computer, desktop computer, etc.) to control the system10. In some such implementations, the user's smart phone and/or tablet can be used as an input device to control the system10by mirroring the display14of the system10on a display of the smart phone and/or tablet and allowing the user to control the system10by touching and/or tapping the smart phone and/or tablet directly. The system10can also include one or more speakers to play music, podcasts, radio, or other audio. The one or more speakers can also provide the user feedback or confirmation of certain actions or decisions.

The system10further includes one or more light sources18. In an implementation, the light sources18are light emitting diodes (LEDs) having variable color and intensity values that can be controlled by the processor22. In other implementations, the light sources18can be incandescent light bulbs, halogen light bulbs, fluorescent light bulbs, black lights, discharge lamps, or any other suitable light source. The light sources18can be coupled to or disposed within the base26of the system10, or they can be coupled to or disposed within the sensor frame28. For example, whileFIG.2only shows two light sources18disposed in a bottom portion of the system10, a plurality of light sources18could be disposed about the frame such that the light sources18generally surround the mirror12. In some implementations, the light sources18may be disposed on either the user-side of the mirror12or the display-side of the mirror12. When disposed on the user-side of the mirror12, the light emitted by the light sources18is configured to travel through the mirror12towards the user. The light sources18can also be rotationally or translationally coupled to the sensor frame28or other parts of the system10such that the light sources18can be physically adjusted by the user and emit light in different directions. The light sources18could also be disposed in individual housings that are separate from both the mirror12and the display14. The light sources18are configured to produce light that is generally directed outward away from the mirror12and toward the user. The light produced by the one or more light sources18can thus be used to illuminate the user (or any other object disposed on the user-side of the mirror12). Because they are variable in color and intensity, the light sources18can thus be used to adjust the ambient light conditions surrounding the user.

The system10also includes one or more cameras20mounted on or coupled to the mirror12. The cameras20could be optical cameras operating using visible light, infrared (IR) cameras, three-dimensional (depth) cameras, or any other suitable type of camera. The one or more cameras20are disposed on the display-side of the mirror12. In an implementation, the one or more cameras20are located above the electronic display14, but are still behind the mirror12from the perspective of the user. The lenses of the one or more cameras20faces toward the mirror12and are thus configured to monitor the user-side of the mirror12. In an implementation, the one or more cameras20monitor the user-side of the mirror12through the partially reflective coating on the mirror12. In another implementation, the one or more cameras20are disposed at locations of the mirror12where no partially reflective coating exists, and thus the one or more cameras20monitor the user-side of the mirror12through the remaining transparent material of the mirror12. The one or more cameras20may be stationary, or they may be configured to tilt side-to-side and up and down. The cameras20can also be moveably mounted on a track and be configured to move side-to-side and up and down. The one or more cameras20are configured to capture still images or video images of the user-side of the mirror12. The display14can display real-time or stored still images or video images captured by the one or more cameras20.

The one or more cameras20are communicatively coupled to the processor22. The processor22can run an object recognition (OR) algorithm that utilizes principles of computer vision to detect and identify a variety of objects based on the still or video images captured by the one or more cameras20. The processor22can be configured to modify the execution of an application being executing by the processor22, such as automatically launching a new application or taking a certain action in an existing application, based on the object that is detected and identified by the cameras20and the processor22. For example, following the detection of an object in the user's hand and the identification of that object as a toothbrush, the processor22can be configured to automatically launch a tooth-brushing application to run on the display14, or launch a tooth brushing feature in the current application. The processor22can be configured to automatically launch an application to assist the user in shaving upon detecting and identifying a razor, or an application to assist the user in applying makeup upon detecting and identifying any sort of makeup implement, such as lipstick, eye shadow, etc. The one or more cameras20can also recognize faces of users and differentiate between multiple users. For example, the camera20may recognize the person standing in front of the mirror12and execute an application that is specific to that user. For example, the application could display stored data for that user, or show real-time data that is relevant to the user.

In an implementation, the processor22can be configured to execute a first application while the display14displays a first type of information related to the first application. Responsive to the identification of the object by the system10, the processor is configured to cause the display14to display a second type of information related to the first application, the second type of information being (i) different from the first type of information and (ii) based on the identified object. In another implementation, responsive to the identification of the object, the processor is configured to execute a second application different from the first application, the second application being based on the identified object.

In some implementations, the components of the system10are generally waterproof, which allows the system10to more safely operate areas where water or moisture may be present, such as bathrooms. For example, a waterproof sealing mechanism may be used between the mirror12and the sensor frame28to ensure that moisture cannot get behind the mirror12to the electronic components. This waterproof sealing mechanism can include a waterproof adhesive, such as UV glue, a rubber seal surrounding the periphery of the mirror12on both the user-side of the mirror12and the display-side of the mirror, or any other suitable waterproof sealing mechanism. Further, a waterproof or water-repellant covering can be placed over or near any components of the system10that need to be protected, such as speakers or microphones. In some implementations, this covering is a water-repellant fabric.

The components of the system10can also be designed to increase heat dissipation. Increased heat dissipation can be accomplished in a number of ways, including by using a reduced power supply or a specific power supply form factor. The system10can also include various mechanisms for distributing heat from behind the mirror12, including heat sinks and fans. In some implementations, the electronic components (such as display14, sensors16, light sources18, camera20, processors22, and memory24) are all modular so that the system10can easily be customized.

FIG.3Aillustrates a front elevation view of the system10, whileFIG.3Billustrates a side elevation view of the system10. As can be seen inFIG.3A, the sensor frame28surrounds the mirror12, while portions of the display14that are activated are visible through the mirror12.FIG.3Aalso shows the two-dimensional grid that can be formed by the sensors16in the sensor frame28that is used to detect the user's finger, head, or other body part. This two-dimensional grid is generally not visible to the user during operation.FIG.3Bshows the arrangement of the sensor frame28with the sensors16, the mirror12, the display14, and the camera20. In an implementation, the processor22and the memory24can be mounted behind the display14. In other implementations, the processor22and the memory24may be located at other portions within the system10, or can be located external to the system10entirely. The system10generally also includes housing components43A,43B that form a housing that contains and protects the display14, the camera20, and the processor22.

FIG.4AandFIG.4Billustrate the system10's ability to allow a user30to view their reflection32in the mirror12, and to view activated segments of the display14behind the mirror12.FIG.4AandFIG.4Balso illustrate the system10's ability to illuminate an object (such as the user's face) with the display14. InFIG.4A, a large segment of the display14is not activated, e.g. the display is off, is showing a black screen, or is otherwise emitting light that cannot be seen by the user through the mirror12. Because the user cannot see any light emitted from the display14in the large segment, the only light that reaches the user30's eyes is light that is incident on the user-side of the mirror12and is reflected back to the user30. Thus, the user30sees a reflection32of themselves and their surroundings due to the light reflected off of the user-side surface of the mirror12. Only in the small segments at the top and bottom of the display14where the display14is activated is the user30not able to see the reflection32, and instead sees the display14.

InFIG.4B, a segment in the center of the display14on the display-side of the mirror12has been activated and is directing light toward the mirror12. In the implementation shown inFIG.4B, the activated segment of the display14has a ring-shaped configuration, and thus the display14shows a ring of light34. The light emitted from the display14is transmitted through the mirror12to the user-side of the mirror12, and thus the emitted right of light is visible to the user30. As can be seen, the light that is transmitted through the mirror12at a given point on the mirror12is stronger than the light that is reflected off of the user-side surface of the mirror12at that point. At those points of the mirror12, the user30only sees the display14through the mirror12, rather than their reflection32in the mirror12. InFIG.4B, the ring of light is transmitted through the mirror12such that it generally surrounds the user30's face. As is shown in the figure, the user30's reflection32in the mirror12is interrupted by the ring of light34transmitted through the mirror12from the display14. The ring of light34can be used to illuminate a face of the user30. Thus, the activated segment of the display is generally aligned with the periphery of the user's face, while an enclosed non-activated segment of the display is generally aligned with the user's face itself, allowing the user to see the reflection of his face.

A method500of illuminating the face of a user is illustrated inFIG.5A. At step502, the one or more sensors16of the system10are activated. At step504, the one or more sensors16monitor the area in front of the user-side surface of the mirror12. The one or more sensors16are configured to detect any portion of the user, such as the user's fingers or face, that the user places within a threshold distance of the mirror12. In an implementation, this threshold distance is less than about five inches, between about 0.1 and four inches, between about 1 and about 3 inches, or less than about 2 inches. The threshold distance could also be greater than about 5 inches. At step506, the one or more sensors16detect a plurality of different points of interaction between the user and the two-dimension grid defined by the sensors16. These interactions are indicative of the user placing his or her face within the threshold distance from the user-side surface of the mirror12. The one or more sensors16recognize that the user has placed their face up to the mirror12rather than a single finger by detecting multiple interactions between the user and the two-dimensional grid defined by the one or more sensors16. In an implementation, if the one or more sensors16detect five points of contact, and that the five points of contact are within less than about 60 square inches, the system10interprets this as the user placing their face within the threshold distance from the user-side surface of the mirror12.

At step508, the system10determines the outline of the user's face from the detected points of interaction with the sensors16. The system10determines the outer boundary that is defined by all of the points of interaction between the user's face and the sensors16and uses this boundary as an estimate of the outline of the user's face. At step510, the electronic display14is activated to outline the user's face with a ring of light. Generally, the display14at this step will show a black screen except for the ring of light, which generally corresponds to the outer boundary of the user's face as detected by the one or more sensors16. The display14could also show a variety of other minor components, such as the time or other icons. Thus, the user looking at the surface of the mirror12will see their reflection, and will also see a ring of light that is being transmitted through the mirror12from the display14. This ring of light generally surrounds the reflection of the user's face in the mirror12and illuminates the user's actual face. The illumination can assist the user in any activity they are about to partake in, such as shaving, brushing their teeth, applying makeup, etc. The system10can also track the user's face in real-time and thus constantly update the position of the displayed ring of light. This allows the user to move their face around while still having their face illuminated at all times by the display14.

An implementation of what is displayed on the electronic display14during method500is illustrated inFIG.6AandFIG.6B. InFIG.6A, the display14primarily shows a black screen. The display14can also show other features, such as the time36, or a lighting icon38. The lighting icon38can show the current state of the light that is being emitted by the light sources18, the current conditions of the ambient light, or both. The lighting icon38may also be interactive to allow the user to adjust the light being emitted by the light sources18. InFIG.6B, the system10has detected and illuminated the user's face by displaying a ring of light34on the display14. As can be seen, the display14in this state includes an activated segment in a ring-shaped configuration substantially surrounding a non-activated segment, such that the non-activated segment is located in the interior of the ring of light34. This allows the user to see his or her own reflection in the mirror12without any interference from the display14. The ring of light34is displayed on the display14at a location that generally corresponds with the periphery of the user's face. This ring of light34is transmitted through the display14and is generally incident upon the user's face, thus illuminating the user's face.

A method700of detecting and identifying an object and modifying an application being executed by the processor22is illustrated inFIG.7. At step702, the one or more cameras20of the system10are activated. At this step, the processor22of the system10will be executing an application and displaying information related to that application on the display14. The application could be a specific program installed in the memory24of the processor22, or could even simply be a basic operating system. At step704, the one or more cameras20monitor the area on the user-side of the mirror12for the user and any objects they may be holding. In an implementation, the images recorded by the camera20could be displayed in real-time on the display14. At step706, the processor22receives images from the one or more cameras20and detects that the user is holding an object. At step708, the processor22identifies the object the user is holding from one of a predetermined number of objects that the processor22can recognize. The processor22of the system10can be programmed with computer vision algorithms that are configured to detect and identify objects that the user is holding. For example, the system10may be configured to detect when the user is holding a toothbrush, a razor, a comb, a makeup implement such as lipstick or eye shadow, a curling iron, a hair straightener, a blow dryer, or any other object that the user may commonly be holding while in front of the mirror12. In some implementations, the user can train the system10to identify different objects. For example, the system10may request that the user hold up a specific object, for example a toothbrush. The system10can then be trained to identify the toothbrush and any characteristics specific to the toothbrush. In this manner, the system10can be trained to improve its recognition of a variety of different objects.

At step710, responsive to and based on the identification of the object the user is holding, the processor22modifies the execution of the application. In an implementation, the modification of the execution of the application includes displaying different information on the display14from what was display prior to the identification of the object. In another implementation, the modification of the execution of the application includes executing a new application that is different from the application that was being executed prior to the identification of the object.

For example, if the system10identifies the object that the user is holding as a toothbrush, the system10can launch a tooth brushing application design to assist people in brushing their teeth. The system10can also start a tooth brushing feature in the currently running application. If the system10identifies a razor, the system10may launch a shaving application or shaving feature in the currently-executing application to assist the user in shaving their face. If the system10identifies any type of makeup implement, the system10could launch a makeup tutorial based on the type and color of makeup that the user is holding. Furthermore, in addition to the use of the sensors16described above with respect to detecting and illuminating the user's face, the one or more cameras20can also be configured to recognize the user's face and/or identify, and take a predetermined action based on this recognition. This action could be illuminating the user's face as described above, or could be some other action.

In other implementations, the system10may identify the object by scanning a product barcode on the object or the object's packaging, or by detecting an RFID tag that is part of or coupled to the object or the object's packaging. To scan the barcode, the system10can include a barcode scanner, or the camera20of the system10can be configured to scan the barcode. To detect the RFID tag, the system10will generally include an RFID reader that is configured to detect and read the RFID tag when the product is near the system10.

In some implementations, the system10can request the user to confirm the system10's identification of an object. For example, the system10may cause the display14to display the name of the object, along with “Yes” and “No” indicators. The user can select one of the indicators to indicate to the system10whether the object has been identified correctly. In other implementations, the user can say “Yes” or “No” out loud, which can be detected by the microphone. The data from the microphone can be sent to the processor to allow the user to confirm whether the system10has correctly identified the object.

A method of compensating for ambient lighting conditions is illustrated inFIG.8. Generally, the system10is able to recognize and compensate for the ambient lighting conditions of the area where the user and the system10are located, so as to illuminate the user with a desired set of lighting conditions. At step802, the system10monitors and detects ambient lighting conditions. In an implementation, the monitoring and detecting of the ambient lighting conditions is done by the one or more cameras20. In another implementation, the system10includes a separate optical sensor that is specifically configured to monitor and detect ambient lighting conditions. The ambient lighting conditions can be determined from sensing the color or hue of an object in the field of view of the camera20/optical sensor with known reflectance properties. The system10could also detect the ambient lighting conditions by capturing an image of the user's face and analyzing the captured image to determine the currently ambient lighting conditions. Further, the system10could detect ambient lighting conditions by using an optical sensor to collect and analyze ambient light. The system10could also use any combination of the above, or other methods, to detect the ambient light conditions.

At step804, the difference between the detected ambient lighting conditions and the desired ambient lighting conditions is determined. The difference can be measured in terms of color temperature/white balance. The desired ambient lighting conditions can be manually input by the user, or the user could also select from one of a plurality of pre-determined desired ambient lighting conditions. In another implementation, the user could use a sensor or other device to measure the ambient lighting conditions at a desired location, for example the user's office, and then upload this measurement to the system10. The system10could then compare the detected ambient lighting conditions to the uploaded ambient lighting conditions. The uploaded ambient lighting conditions could be in the form of simple numerical data, but could also comprise an image of the desired location with desired lighting conditions. At step806, the system10adjusts the color, intensity, and other parameters of the one or more light sources18to compensate for the detected ambient light conditions and produce the desired lighting conditions. This desired lighting condition can be achieved causing the one or more light sources18to emit various shades and intensities of white light. The desired lighting conditions could also be achieved by causing the one or more light sources18to emit different colors of light, such as but not limited to red, green, blue, purple, yellow, orange, etc. The illumination of the object due to the ambient lighting conditions and the illumination of the object due to the light produced by the one or more light sources18combine to cause the object to be illuminated by the desired lighting condition.

In some implementations, the system10continually detects the ambient lighting conditions, adjusts the light being emitted from the light sources18, newly detects the ambient lighting conditions, and then readjusts the light being emitted from the light sources. The system10can also use the detected ambient lighting conditions to calibrate itself, such that each time the system10is activated by the user, the system10will automatically adjust the light sources18to bring about the desired ambient lighting conditions. The system10can also allow the user to monitor the process and view the updated lighting conditions, either by the user's reflection in the mirror12, or by displaying a live feed from the one or more cameras20on the display14. The user can also manually adjust the light being emitted from the one or more light sources18depending on his or her preferences.

The system10can monitor and adjust the ambient light to improve pictures that the user may take, of themselves or other objects. The system10can also provide feedback to the user to improve pictures, such as suggesting that the user move themselves or the object, suggesting the user move the camera20, or automatically moving the camera20.

A method900of capturing an image of the user is illustrated inFIG.9. At step902, the one or more cameras20are activated and begin monitoring the area on the user-side of the mirror12. At step904, a real-time image of the user's face is shown on the display14in an area of the display14that generally aligns with the user's face. The user thus sees the real-time image of themselves from the one or more cameras20, rather than their reflection. At step906, one or more user-selectable icons are displayed on the display14. Each of the user-selectable icons corresponds to a different ambient lighting condition, such as natural light, no light, soft light, sunset, sunrise, rainbow colors, or other desired ambient lighting conditions. At step908, the system10detects user input that corresponds to a selection of one of the user-selectable icons. At step910, the system10adjusts the color, intensity, or other parameter of the light emitted by the one or more light sources18so as to illuminate the user with the desired ambient lighting condition. At step912, an image of the user's face and/or other portions of the user is captured.

In another implementation, the system10recognizes the user's face when the user steps into the field of view of the one or more cameras20. Instead of displaying user-selectable icons, the system10automatically selects a pre-determined lighting profile based on which user the system10has recognized. For example, a certain user may prefer to capture images of themselves using the system10in soft light, and thus can configure the system10to automatically illuminate their face in soft light. The capture of the image can be manually triggered by the user, or can be automatic once the desired lighting conditions have been achieved. The system10can also give a countdown to the user so they can be prepared for the image to be captured.

An implementation of what is displayed on the electronic display14during method900is illustrated inFIG.10. InFIG.10, the display14shows a plurality of icons40A,40B,40C, each corresponding to a different ambient light condition that the user may select. Icon40A corresponds to no light. Icon40B corresponds to natural light. Icon40C corresponds to soft light. The user can select any of these icons to cause the processor to adjust the light produced by the light sources such that the ambient light conditions in the area surrounding the user correspond to the selected icon. The display14can also display the time36, or a lighting icon38that indicates the lighting conditions and can allow the user to adjust them. In an implementation, the display can show instructions to the users. In another implementation, the display shows a real-time image of the user captured by the camera20.

Referring now toFIG.11A, an implementation of the system10includes a first display14A and a second display14B. The first display14A and the second display14B are both located on a side of the mirror12opposite where a user would be located. The first display14A and the second display14B are generally spaced apart such that a gap is defined between them. The system10further includes one or more cameras20that are located on the same side of the mirror12as the first display14A and the second display14B. However, the one or more cameras20are located in the gap between the first display14A and the second display14B, rather than being up near the top of the mirror12. The system10will generally include a sensor frame28including one or more sensors16, and one or more light sources. The system10thus operates in generally the same manner as other implementations disclosed herein. However, by locating the one or more cameras20between the first display14A and the second display14B, the one or more cameras20will generally be positioned level with a user's face, and can thus obtain images of the user at a more appropriate viewing angle.

FIG.11Billustrates an implementation of the system10that includes two displays14A and15B, and two cameras20A and20B. As shown, this implementation of the system10includes a left camera20A and a right camera20B. The field of view of the left camera20A generally spans from the center of the user's face to the far left boundary of the user's face (e.g., the outer edge of the user's left ear) so as to capture an image or a video of the left side of the user's face. Similarly, the field of view of the right camera20B generally spans from the center of the user's face to the far right boundary of the user's face (e.g., the outer edge of the user's right ear) so as to capture an image or a video of the right side of the user's face. The images and/or videos from the two cameras20A,20B can be combined to form a single front-facing image of the user's face. In some implementations, there is a slight overlap in the fields of view of the two cameras20A,20B near the center of the user's face to ensure all of the user's face is captured by at least one of the cameras20A,20B.

FIG.12illustrates an implementation of the system10where the mirror12and the sensor frame28have different shapes. In this implementation, the mirror12is oval-shaped while the sensor frame28is rectangular-shaped.FIG.13illustrates an implementation of the system10where the mirror12and the sensor frame28have the same shapes but different sizes. Both the mirror12and the sensor frame28are rectangular-shaped, but the mirror12is larger than the sensor frame28. In this implementation, the sensor frame28is generally aligned with an area of the mirror12interior to the periphery of the mirror12.

Other implementations of the smart mirror system10are contemplated in accordance with the present disclosure. For example, the system10could operate without the mirror12. The images captured by the camera20could be displayed on the display device to take the place of the reflection in the mirror12.

In some implementations of the present disclosure, the system10connects (e.g., wirelessly, via one or more wired connections, or a combination thereof) with one or more other devices. By connect with, it is meant that the system10can receive data from the one or more other devices and/or transmit data and/or instructions to the one or more other devices. For example, the system10can connect with one or more mobile phones, one or more laptop computers, one or more desktop computers, one or more online servers, one or more fitness trackers, one or more scales (e.g., Internet-connected scales), one or more cloud services (e.g., a backend of an application-enabled smart device), one or more Internet-connected water bottles, one or more Internet-connected thermostats, one or more smart watches, one or more smart speakers, any other smart device, or any combination thereof.

In some implementations, the system10aggregates all of the data collected from any other devices connected to the system10and provides an easy-accessible location and/or portal for the user to view and/or manipulate and/or analyze the aggregated data. Thus, system10can act as a centralized hub (e.g., a base station) for all of the devices and/or services that the user is currently using. Such a centralized hub can collect data from any services the user is using and/or subscribing to, by communicating with any devices associated with these services (e.g., by communicating directly to the device via a wireless connection or wired connection and/or via a connected cloud service that receives the data from the respective device). In some implementations, the data can be stored locally by the centralized hub for the user to access. In additional or alternative implementations, the system10can send the data to the respective back-ends of the services.

In some implementations, the system10analyzes all or some of the collected/aggregated data from the other devices and correlates different events. In some such implementations, the system10offers advice to the user based at least in part on the analyzed aggregated data. For example, the system10obtains data from the user's Internet-connected scale. The system10analyzes the data and determines that the user lost weight in a given time period. In some implementations, in view of the weight loss determination, the system10analyzes data from one or more of the other connected devices that generate data during a corresponding time period of the weight loss to determine what other activities and/or events the user experienced that may have contributed to the weight loss. For example, the system10may recognize that during the relevant time period where the user lost the weight, the user also drank a certain amount of water, slept a certain number of hours per day, underwent a certain amount of activity day, or any combination thereof. The system10can correlate all of this data, present it to the user, and offer advice on what may have led to the user's weight loss and what may lead to further weight loss or gain.

The connection of the system10with the other devices allows the system10to utilize any data or portion of data from one or more of the other connected devices and/or services in a manner that is not normally possible. This is because such data from one of other connected devices and/or services is not generally accessible by another of the other devices and/or services. Thus, the system10aids in integrating separate and distinct platforms that have not communicated in the past.

In an implementation, the system10can monitor features on their body or face, such as moles, wrinkles, or beauty spots. The system10captures an initial image of the user using the camera20. The user can view the captured image and select one or more features for the system10to monitor. Once the feature is selected by the user the system10, using image processing and/or computer vision algorithms, can allow a finer selection or delineation of the feature selected by the user. In another implementation, the system10can mark features without user selection. Once the features are selected and marked by the system10, the initial location, color, and other characteristics of the selected features are stored in the memory of the system10. The system10can capture subsequent images of the user and identify the selected features. Any deviation in the characteristics of the features can be monitored by the system10. Any unusual deviations can be reported by the system10to the user. In some implementations, the system10can monitor features such as moles, growths, sores, or other features that can appear on the user's skin that may be indicative of skin cancer or other diseases. The system10can monitor features by looking at images of only the feature itself, or can monitor the features by analyzing images of the user's entire face and/or body. The system10can also monitor other features indicative of medical conditions or diseases. For example, the system10can monitor the user's hair to determine if any of the user's hair is falling out. The system10can also monitor the user's teeth, gums, or lips for any indication of cavities, gum diseases, or other afflictions. The user's eyes can also be monitored for any type of distinguishing feature.

Referring now toFIG.14, in some implementations of the system10, the mirror12includes an inner zone13and two outer zones15A and15B. The inner zone13defines the area on either surface of the mirror12(or both) that includes the partially reflective coating. The outer zones15A and15B define the area on both surfaces of the mirror12where no partially reflective coating exists. In some implementations, the outer zones15A and15B are formed as generally vertical bands or strips. In these areas, the mirror12comprises only the generally transparent substrate material (e.g., glass, acrylic, etc.). In some implementations, one or more light sources (such as light sources18discussed herein) can be positioned on the display-side of the mirror12. The light sources are positioned such that they emit light towards the mirror12, which propagates through the generally transparent substrate material and then to the user. The light sources can thus shine light through the generally transparent substrate material of the mirror12to illuminate the user, the area or objects surrounding the user, or both.

In some implementations, one or both of the display-side surface of the mirror12and the user-side surface of the mirror12within the outer zones15A and15can be sandblasted. Because the partially reflective coating is not present on either surface of the generally transparent substrate material within the outer zones15A and15B, the generally transparent substrate material itself is sandblasted. By sandblasting the surface of the substrate material, the light emitted toward the user by the light sources is diffused as it travels through the substrate material. By diffusing the light that is emitted through the mirror12towards the user, the system10achieves a more even illumination of the user and the area/objects surrounding the user. Further, if the outer zones15A and15B were both sandblasted and contained the partially reflective coating, the resulting light transmitted by the light sources towards the user would generally be dull and not provide sufficient illumination. Thus, the outer zones15A and15B are generally only sandblasted. However, in certain implementations, the outer zones15A and15B, or the inner zone13, may be sandblasted and include the partially reflective coating.

In one implementation, the partially reflective coating is present only on the surface of the user-side of the substrate material within the inner zone13, while the substrate material is sandblasted only on the display-side within the outer zones15A and15B. By coating only the user-side surface of the substrate material with the partially transparent coating, images designed to be shown to the user through the mirror12by the displays on the display-side of the mirror12are more visible. Further, if the user-side surface of the substrate material within the outer zones15A and15B is sandblasted, the resulting porous surface can more easily absorb liquids or other substances such as shaving cream, toothpaste, etc. By only sandblasting the display-side of the substrate material, the user-side of the substrate material within the outer zones15A and15B will generally not absorb undesirable substances. The partially reflective coating is not present on the user-side surface of the substrate material within the outer zones15A and15B so as to not dull the light that is emitted by the light sources and transmitted through the mirror12.

In another implementation, the partially reflective coating is present only on the surface of the display-side of the substrate material within the inner zone13, while the substrate material is sandblasted only on the user-side within the outer zones15A and15B. In other implementations, the partially reflective coating is present on the same side of the substrate material that is sandblasted. In these implementations, either or both of the user-side and the display-side of the substrate material is coated by the partially reflective coating within the inner zone13. The outer zones15A and15B of that same side of the substrate material is sandblasted. In some implementations, the partially reflective coating is deposited on the entirety of the desired surface(s) of the substrate material, including in the inner zone13and in the outer zones15A and15B. The partially reflective coating within the outer zones15A and15B can then be removed, either by the sandblasting process or by another process preceding the sandblasting process. In still other implementations, the partially reflective coating is deposited on the desired surface(s) only within the inner zone13.

Referring now toFIG.15, the system10can include an obstruction100that is configured to at least partially block or obscure the field of view of the camera20. This allows the user to prevent him or herself from being viewed by the camera20, and also prevents the camera20from viewing the area surrounding the user. In some implementations, the user can manually move the obstruction100between a stored position and a deployed position. In other implementations, the obstruction100can be moved between the stored position and the deployed position using one or more electronic actuation devices, which can include a solenoid, a spring, a lever, a magnet, any combination thereof, or any other suitable device.FIG.15illustrates the obstruction100in the deployed position. In other implementations, the processor(s) of the system10are configured to move the obstruction100between the stored position and the deployed position. As shown in the implementation ofFIG.15, the obstruction100can be mounted to the underside of the top portion of the frame, and includes a portion projecting downwardly that blocks the field of view of the camera20. In some implementations, the obstruction100is pivotally coupled to the frame. In other implementations, the obstruction100is slidably or translatably coupled to the frame.

FIGS.16A and16Bshow perspective views of the top and bottom of the obstruction100. As shown, the obstruction100generally includes a baseplate102and a projection portion104. The baseplate102includes a variety of fastener apertures106A-106H that are used to secure the obstruction100to the frame (or another component) of the system10. In some implementations, respective fasteners extend through the fastener apertures106A-106H and into the frame to secure the obstruction100to the frame. The fasteners can be screws, nails, pins, rods, clips, etc. In other implementations, the obstruction100can be secured to the frame without the use of fasteners and the fastener apertures106A-106H, for example by using adhesive.

The projection portion104is coupled to the baseplate102in a manner that allows the projection portion104to move between a stored position and a deployed position.FIGS.16A and16Bshow the projection portion104in the deployed position. In this deployed position, the projection portion104extends away from the baseplate102so as to block the field of view of the camera20. In some implementations, the projection portion104is pivotally coupled to the baseplate102via a hinge or other structure that allows the projection portion104to pivot relative to the baseplate102. In still other implementations, the projection portion104is configured to move generally along one dimension between the stored position and the deployed position. The baseplate102can include a depression108defined therein that is sized to accommodate the projection portion104when the projection portion104is moved to the stored position. The projection portion104and the depression108can be sized so that the obstruction100has a generally flat surface when the projection portion104is in the stored position.

In some implementations, the projection portion104can be biased towards either the stored position or the deployed position, or can be selectively biased towards both positions. In still other implementations, the projection portion104is not biased in either position. The projection portion104can be biased using any suitable mechanism, such as a spring. When the projection portion104is biased towards only one of the stored and deployed positions, the obstruction100generally include a retention feature that retains the projection portion104in the non-biased position. The retention feature could be a clip, strap, or other similar feature. The retention feature could also include a structure on the projection portion104that snaps into a corresponding depression on the baseplate102.

In some implementations, the retention feature is the depression108, which may be sized so as to retain the projection portion104within the depression108via a friction fit. For example, the portion of the baseplate102that forms the outer periphery of the depression108may include a small semi-resilient ridge. The semi-resilient ridge can form a circle with a diameter that is slightly less than the diameter of the projection portion104. When the projection portion104is moved to the stored position, the force imparted to the projection portion104can overcome the friction force between the projection portion104and the semi-resilient ridge so that the projection portion104is seated in the depression108and retained there.

When the projection portion104is selectively biased in both the stored position and the deployed position, the projection portion104must be moved a certain amount away from either position until the bias alternates, which causes the projection portion104to continue to move towards the other position. For example, if the projection portion104is selectively biased towards both positions and is currently in the stored position, the user (or a component of the system10) must begin to move the projection portion104away from the baseplate102. Once the projection portion104is moved a certain amount away from the baseplate102and reaches an inflection point, the bias alternates towards the deployed position. Moving the projection portion104from the deployed position to the stored position also occurs in a similar fashion. In some implementations, the inflection point may be about halfway between the stored position and the deployed position. In these implementations, no retention feature is necessary, as the projection portion104is always biased towards the position in which it is currently stored.

In the implementations where the projection portion104is not biased towards either position, the obstruction100still generally includes a retention feature to prevent gravity from causing the projection portion104from moving between positions. For example, the projection portion104may still snap into the depression108in the baseplate102so as to prevent the projection portion104from moving towards the deployed position due to the influence of gravity. In these implementations, friction in the structure used to couple the projection portion104to the baseplate102may retain the projection portion104in either position. For example, friction in the hinge or other pivoting mechanism used to pivotally couple the projection portion104to the baseplate102may impart a force on the projection portion104that is greater than the force imparted on the projection portion104due to gravity. This causes the projection portion104to remain in the stored position, even when gravity is imparting a force on the projection portion104towards the stored position.

The obstruction100can be formed from an opaque material that does not let any light pass therethrough, or a partially transparent material that only lets a portion of light therethrough. For example, the obstruction100could be formed of one or more optical filters that are configured to block or lass different wavelengths of light.

In different implementations, the camera20can have various positions relative to other components of the system10. In some such implementations, the camera20is located on the user-side of the mirror12, such that the mirror12is positioned between the camera20and the user. These implementations are illustrated inFIGS.3B,11A, and11B. In these implementations, the camera20can be positioned above the display14, as shown inFIG.3B. The camera20could also be positioned between separate display portions14A and14B, as shown inFIGS.11A and11B. In other implementations, the camera20could be located below or to the side of the display14or display portions14A and14B.

In still other implementations, the camera20is located in an area such that the mirror12is not positioned between the camera20and the user. In these implementations, the camera20has a direct line of sight to the user such that substantially all of the light entering the camera20does not first pass through the mirror12. In some of these implementations, the camera20is positioned in front of and/or on the mirror12such that the camera20is positioned between the mirror12and the user. The camera20can still be within the frame28of the system10, and can generally be located in any direction relative to the display14, e.g., above, below, or to the side of the display14. In some implementations, the camera20is positioned outside of the frame28of the system10. In these implementations, the mirror12is not positioned between the camera20and the user, and the camera20can be positioned on either the display-side of the mirror12or the user-side of the mirror12. In still other implementations, the camera20is positioned on the internal side of the mirror12; however, the portion of the mirror12that light passes through when received by the camera20(e.g., the portion of the mirror12that the camera20“looks” through) can have little or no reflective coating on the substrate material. The camera20will thus effectively view the user and the surrounding area through the generally transparent substrate material and not through a mirrored/reflective film, thereby improving the quality of the images generated by the camera20.

While the mirror12is generally depicted as being a full-size rectangular mirror in which the reflection of a substantial portion of the user30's upper body fits and is visible, the mirror12can have any desired size and/or shape. For example, the mirror12can be a relatively smaller mirror typically used to apply makeup or to shave, a handheld mirror that can be stored in a purse and//or pocket, a relatively larger mirror that shows the reflection of the user30's head and neck, or any other suitable size. In some implementations, the mirror12can have a generally rectangular shape, with a long dimension that is between about four inches and about twelves inches. In other implementations, the mirror12can be circular, with a diameter of between about four inches and about twelve inches. In still other implementations, the mirror12has a square shape that is three inches by three inches, four inches by four inches, five inches by inches, or any other suitable size. In still other implementations, the mirror12has a size and shape designed to match the size and shape of the display14, and various devices can be used as the display14. For example, the display14could be a handheld device (such as the user30's smartphone), and the mirror12can have a size similar to that of the handheld device. The mirror12can have a size and shape similar to a tablet or a laptop, so that the mirror12can be used with a tablet computer or a laptop computer. In still other implementations, an LED dot matrix display can be used as the display14, and the mirror12can be sized to be used with the LED dot matrix display. Thus, the mirror12and display14can be used as a portable and/or modular device that the user30can use for a variety of different applications or in a variety of different locations.

As discussed herein, the system10is generally configured to recognize objects in the field of view of the camera20and launch and application or tutorial based on the identified object. In some implementations, this tutorial can show the user how to apply a certain product, which could be makeup (e.g. lipstick, eyeliner, etc.), hair dye, toothpaste (e.g. the user brushing their teeth), or any other product that the user may apply in front of the mirror12.

FIGS.17A and17Billustrate an implementation of the mirror12and the display14when the system10identifies an object or product held by the user using the object recognition algorithm, and launches a tutorial based on that object or product. As shown, the display can show the time36, and can also show information37relating to the user's schedule. In the illustrated implementation, the information37includes an indication of when the user is supposed to leave for their destination. This information can be based on the user's calendar, which can be stored in the memory of the system10, or synced from another device or location. The display also shows a variety of icons, that may or may not be interactive. As discussed herein, the display14can display the lighting icon38. The display can also display a weather icon42can show an image that is indicative of the current weather conditions, such as clouds, a sun, snow, etc. An audio player icon44is interactive and can be operated by the user to play music, podcasts, radio stations, or other audio sources.

As shown inFIG.17A, the system10shows an image45that contains the user46and the object48being held by the user46. The image45can be a reflection in the mirror12, or can be a still image or real-time video feed shown on the display14. When the image45appears as a reflection in the mirror12, the display14is inactivated in the center where the reflection of the user46and the object48appears. When the object48is in the field of view of the camera20, the system10can detect and identify the object48and (if applicable) the product contained therein.

FIG.17Bshows the state of the system10upon detecting that object48that the user46is holding the bottle of eyeliner. As shown, the display can display an image50of the eyeliner (which can be a stock image or an image captured by the camera20) and a description53of the object48or the product contained therein. The system10can again show an image45of the user46, either as a still image or real-time video feed on the display or as a reflection in the mirror12. The display also shows an initiation icon52that can be activated by the user46to launch a tutorial application, which can show the user46how to apply the eyeliner. The initiation icon52can include the words “START TUTORIAL,” or can show other text or images. Finally, the display shows a cancellation icon54that the user46can interact with to exit this stage if they do not wish to proceed with the tutorial. Activating the cancellation icon54causes the system10to revert back to the previous stage. This can be useful when, for example, the system10incorrectly identifies the object48that the user is holding, or if the user46changes their mind and no longer wishes to use the object48.

FIGS.18A,18B, and18Cillustrate the steps of the tutorial application that is executed by the system10. The tutorial can be launched responsive to the user interacting with the initiation icon52that is shown inFIG.17B. The tutorial can also be launched via another method, such as voice activation. As illustrated inFIGS.18A,18B, and18C, the image45of the user46is shown as either a reflection in the mirror12or as a real-time video feed shown on the display. The display also shows an image frame56from a live action video showing how to apply the eyeliner. An interactive icon58can be overlaid on the image frame56. When the user46interacts with the icon58, the display plays the video for the user46. The video may be stored in the memory of the system10, or can be downloaded from another location and played in response to the user activating the icon. The video plays can play on a loop, or can play a set number of times. WhileFIGS.18A,18B, and18Cshow the icon58overlaid on the image frame56during the tutorial, the live action video can be played repeatedly during the tutorial. During the tutorial application, the display can also show a magnification icon60that initiates a magnification feature when activated by the user. The magnification icon60can include text such as the word “MAGNIFIER,” or could also be displayed as an image.

The tutorial application can also display an image or a series of images showing the steps to apply the eyeliner. In one implementation, the display displays an animated GIF (Graphics Interchange Format) that shows the steps to the user.FIG.18Aillustrates the display showing a first frame62A of the GIF that demonstrates the first step of the application process.FIG.18Billustrates the display showing a second frame62B of the GIF that demonstrates the second step of the application process.FIG.18Cillustrates the display14showing a third frame62C of the GIF demonstrating the third step of the application process.

Each of the frames62A-62C is indicated with a number and other markings showing the which action to take. For example, each step inFIGS.18A,18B, and18Cincludes an arrow64A,64B, or64C that indicates to the user which motion to take to apply the eyeliner and achieve the desired result. During the tutorial, the display can repeatedly show each frame62A-62C of the application process in sequential order until the user46finishes applying the eyeliner. The image or series of images showing the steps to apply the eyeliner can take the form of other file types as well. For example, instead of showing a GIF, the display can instead show a series of individual still image files, such as JPEG files or PNG files. In some implementations, the display shows an animated video instead of an image or series of images. Generally, the GIF presents to the user a looping video that illustrates an animation associated with the tutorial, e.g., showing how to applying the product.

As shown inFIGS.18A,18B, and18C, the display14during the tutorial application can still display the time36, the information37related to the user46's schedule, the lighting icon38, the weather icon42related to the current weather conditions, and the audio player icon44allowing the user to play music or other audio.

In some implementations, the system10can assist the user46in conducting activities (applying eyeliner) by highlighting portions of the user46's face where they need to apply the product. In one example, the light sources of the system10can emit light onto a portion of the user46(e.g. their face) to highlight where to apply the product. The light sources can include LEDs, lasers, or other suitable types of light-emitting devices. The light emitted by the light sources can have any suitable shape or form, such as dots, circles, squares, triangles, curves, arcs, arches, lines, any combination thereof, or any other suitable shape. The light projected on to the user46's face will be visible in the image45of the user46, either as a reflection in the mirror12or as an image on the display.

In another implementation, the display is used to guide the user46and to overlay template features onto the user46's face to aid the user in conducting activities. For example, when the image45of the user46is a real-time image on the display, the display could highlight portions on the user46's face where to apply the product. In another implementation, the image45of the user46could be a reflection in the mirror12, but the display could still highlight any portions on the use46's face where to apply the product. The light in this location from the display will be brighter than the light reflecting off of the mirror12at this location, and thus the user46will be able to see the light indicating where they are supposed to apply the product. The template features that can be shown by the display can take any suitable shape or form, such as dots, circles, squares, triangles, curves, arcs, arches, lines, any combination thereof, or any other suitable shape.

FIGS.19A and19Bshow the state of the system10when the user initiates the magnification feature by selecting the magnification icon60as shown inFIGS.18A-18C. The display can continue to show the image frame56of the video and the icon58that can be activated to play the video, or can continue to show the video itself. The display also continues to show the steps of the looping video illustrating the animation associated with the steps for applying the eyeliner (e.g. the GIF). In the first stage of the magnification feature illustrated inFIG.19A, the display displays a full-size image66of the user46's face, along with a caption68. The caption68can include the instructions “Tap anywhere to zoom in!” While during the tutorial the system10can show the user46's face as just a reflection in the mirror12, once the magnification feature is activated, the user46's face is shown as an image on the display.

FIG.19Bshows a second stage of the magnification feature. Once the user46taps or otherwise interacts with the image66of their face at an area of interest, the display shows a zoomed-in image70of the user's face that includes that area of interest. For example, as illustrated inFIG.19B, when the user is applying eyeliner to the user's eyelashes, the user can zoom in on the portion of their face that includes their eyelashes. This allows the user to better view their eyelashes as they apply the eyeliner. The user can also manipulate the zoomed-in image70by interacting with the display. The user can swipe or drag the zoomed-in image so that different areas of the user's face. The user can also tap again to zoom back out. The display can show instructions72that indicate to the user how to manipulate the zoomed-in image70. In some implementations, the system10is configured to track the area of interest as the user moves. Thus, the display shows the zoomed-in view of the area of interest, and continues to display the zoomed-in view of the area of interest as the user moves. The area of interest could be the user's eyes, noses, ears, mouth, teeth, etc., or any other feature on the user's face or body.

FIGS.20A-20Fillustrate a tutorial for applying hair dye that the system10can execute for the user. During the tutorial, the display can show the same information and icons discussed above, including the time36, information37relating to the user's schedule, the lighting icon38, the weather icon40, the audio player icon41, and the cancellation icon54. At a first step in the tutorial as shown inFIG.20A, the display can show a list73of items the user will need to apply the hair dye. This list73is be displayed after the system10identifies the object that the user is holding as hair dye. The display can also show the initiation icon52that the user can interact with to begin the tutorial. The display may also show a warning74to the user related to the time needed to complete the application of the hair dye. For example, if the user has to depart in a certain amount of time period (which is communicated to the user via the information37on the display) but it normally takes longer than that time period to complete the application of the product, the system10can issue a warning74to the user indicating this fact.

Once the user initiates the tutorial by interacting with the initiation icon52, the display can show a number of images to the user showing them how to apply the hair dye.FIG.20Billustrates the display displaying an image76A to the user showing how to mix and prepare the hair dye, along with instructions78A for mixing and preparing the hair dye.FIG.20Cillustrates the display displaying an image76B to the user showing how to protect their skin and clothing from the hair dye, along with instructions78B for protecting their skin and clothing from the hair dye.FIG.20Dillustrates the display displaying an image76C of a user preparing their hair for the dying process, along with instructions78C for preparing the user's hair for the dying process.FIG.20Eillustrates the display displaying an image76D of a user applying the hair dye to their hair, along with instructions78D for applying the hair dye. Finally,FIG.20Fillustrates the display displaying an image76E of a user rinsing the hair dye out of their hair, along with instructions78E for when and how to rinse the hair dye out of their hair. At this last step shown inFIG.20F, the display can also show a timer80that indicates how much longer they need to keep the hair dye in their hair before the rinse it out. The system10may also include an alarm to indicate to the user when the timer80has reached zero and they can begin to rinse the hair dye from their hair. The alarm could be an audio alarm, a visual alarm, a combination thereof, or any other suitable type of alarm.

As shown inFIG.20E, the instructions78D for applying the hair dye may also include a suggestion to use a 360° visualization feature. This is so that the user can ensure that they have applied the hair dye evenly on their entire head, including the back and sides. At this step, the display displays an icon82that the user can select to launch the 360° visualization feature, which is shown inFIGS.21A and21B. Once the visualization feature is launched, the camera20can begin recording the user, e.g., the camera20can begin generating image data that is reproducible as an image(s) or video of the user. At this point, the display can show a live video feed88of the user, along with text instructions84and pictorial instructions86. The text instructions84can, for example, instruct the use to being turning around so that the camera20can record their head. The pictorial instructions86can include a looping arrow showing the user which direction to rotate. Once the recording has been completed, the display can display a 3D model90of the user based on the recorded video, as shown inFIG.21B. This 3D model can be manipulated by the user to rotate and/or zoom in on various portions of the user's head.

WhileFIGS.18A-21Bshow tutorials for eyeliner and hair dye, the system10can provide tutorials that are generally applicable to any type of product or application thereof. For example, the system10may provide tutorials for the application of makeup, application of moisturizer, application of hair dye, application of hair gel, application of shaving cream, shaving, brushing teeth, eyebrow waxing, waxing, eyebrow threading, threading, facials, ear cleaning, hair styling, application of contact lenses, application of facial masks, or any combination thereof. Moreover, any of the features described with respect to these figures or other figures can be utilized on conjunction with any type of tutorial, or for other non-tutorial applications that the system10may execute. For example, the 360° visualization feature can be used in conjunction with any taken by the user or application executed by the system10, not just for a hair dying tutorial.

Referring now toFIGS.22A-22H, the system10can also execute applications that allow the user to view modified images of themselves based on the modification of different characteristics of the user. For example, the user can view images of themselves that show what they may look like after the application of different types of products or procedures, for example applying different hair dye or getting different hair styles. As part of this process, the system10can capture a “before” image, as shown inFIG.22A. Here, the display can display a real-time video feed200on the display that includes the user. The display can also include a centering indicia202that shows the user where to position themselves to capture a high-quality before image. The purpose of the centering indicia202is to ensure that any images that the system10captures of the user all show the user in the same location. The display also displays instructions204for capturing the before image. Once the user instructs the system10to capture the before image (for example via physical interaction with the display or via voice activation), the display can show the before image206as shown inFIG.22B. The display also displays an initiation icon52that allows the user to begin selecting different types of products or procedures to apply.

As shown inFIG.22C, the display can show a plurality of user-selectable options210A-210H. Each of the user-selectable options210A-210H are associated with a unique modification of a first characteristic related to the user. In this example, that characteristic is hair color, and each individual user-selectable option210A-21H is associated with a unique proposed modification of that characteristic, e.g. different hair colors. Each of the user-selectable options210A-210H can also include an image showing the different proposed modifications. In some implementations, the images may show an image of a person with that proposed modification. In other implementations, the images may show some sort of indicia representing the proposed modification.

Along with the user-selectable options210A-210H, the display can show the before image206. Once the system10receives via an input device a selection from the user of one of the options210A-210H, the display can show a modified image208that shows the user with the proposed modification of the characteristic corresponding to the selected option. In this example, the modified image208shows the user with the hair color corresponding to the selection option. In some implementations, the display shows a real-time video feed of the user that is modified in real-time as different modifications (e.g. hair colors) are selected by the user. If the user likes any of the proposed modifications, they can tap or interact with the icon212to note that. Once the user has identified at least one proposed modification that they like, they can tap the confirmation icon214to proceed to the next stage. In other implementations, the system10can display a prior image of the user captured the last time the user modified the first characteristic the same as a selected option. This allows the user to check their own history to determine which modification they like.

In some implementations, instead of displaying the options210A-210H for the user to select, the system10can identify an object that the user is holding and show the modified image208based on that identification. For example, the system10can determine that the user is holding a certain color hair dye, and can produce the modified image208that shows the user with hair the color corresponding to the identified hair dye.

As shown inFIG.22D, the display can then display a second set of user-selectable options218A-218E that correspond to unique proposed modifications of a second characteristic related to the user. In this implementation, the second characteristic is hair style. Each of the second set of options218A-218E can include an image showing the proposed modification, or some other indicia. As the user selects different hair styles, the display can show an image216of a person with that hair style. Generally, depending on what the second characteristic of the user is, the system10may not be able to show a still or real-time image/video of the user with the proposed modification to that characteristic. For example, it may be difficult for the system10to show a modified still image or real-time video of the user with a different hair color. However, the system10can instead just show stock images or videos of a person with that hair style.

After the user has selected at least one of options218A-218E to the second characteristic using the icon212, the user can interact with the icon220to move to the stage illustrated inFIG.22E. Here, the display can show the before image206of the user, along with a plurality of images222A,222B,222C that show the user in their before image206but with the selected modifications to the first characteristic. The display can also show images224A,224B,224C that correspond to the selected modifications to the second characteristic. The images224A,224B,224C can be images of other people. After reviewing the options for modification of the first and second characteristic, the system10can display an image226showing the desired modification to the first characteristic and an image228showing the desired modification to the second characteristic, as shown inFIG.22F.

As illustrated inFIG.22G, once the user has actually undertaken the selected modifications to the characteristics (e.g. they have dyed and cut their hair), the system10can obtain an “after” image230. As shown, the display can show the real-time video feed200and the centering indicia202. The centering indicia202ensures that both the before image206and the after image230will show the user from the same angle. Once the system10captures the after image230, the system10can show the before image206and the after image230side-by-side to allow the user to compare them. The system10may also be able to use the before image206and after image230to train itself to better show the proposed modifications to the user's characteristics. For example, the system10can compare the image it generated to show a potential modification with the actual image of the user after they have undergone that modification. The system10can identify any differences and use that to improve its generation of the modified images in the future.

The features discussed in connection withFIGS.22A-22Hcan generally be used in connection with any type of activity and generally in any combination. For example, these features can be used with activities other than cutting or dying the user's hair, such as applying makeup. Moreover, for any characteristic of the user, the display can show images that illustrate a variety of different modifications to that characteristic. The images can either be modified images of the user him or herself, or can be images of other people showing the modification.

The system10can also maintain a photo history of the user's activities. In one implementation, the display can show the before and after images to the user after they complete the activity (e.g. applying the makeup or other substance) so they can examine how well they performed the activity. In another implementation, the display can show to the user past results from applying makeup or other substances. This can allow the user to examine their past applications and determine whether they made any errors that they need to try and avoid. This also allows the user to check their history to determine which types of activities they might want to repeat (e.g., a past haircut that the user would like to have again).

Referring now toFIG.23, the system10is configured to allow the user to adjust the light emitted by the light sources via taps or swipes. In an example, the display shows instructions232to the user for adjusting the color. Horizontal icons234A and234B can be manipulated (by tapping on the icons or swiping across the icons) to adjust a property of the light, such as brightness, warmth, color, temperature, etc. Vertical icons236A and236B can also be manipulated in the same fashion. In some implementations, horizontal icons234A and234B can be used to adjust a first property of the emitted light (e.g. brightness), while the vertical icons236A and236B are used to adjust a second property of the emitted light (e.g. color).

In some implementations, the display14displays a virtual power button to the user when in a rest mode or an off mode. When the user interacts with the virtual power button, the system10wakes up or turns on. The user can interact with this virtual power button by touching the surface of the mirror12(or moving their finger into close proximity with the surface of the mirror12) where the virtual power button is displayed. In other implementations, the system10includes a physical power button that the user may interact with. In still other implementations, the system10includes a proximity sensor that is configured to detect when the user has placed his or her finger near within a threshold distance of the user-side surface of the mirror12for at least a specified time period. Detection of this causes the system10to wake up or turn on.

In some implementations, the system10can execute an application to test the user's eyesight or other properties related to the user's eyes. This application can be launched manually by the user. However, the system10may also launch the application automatically upon detecting that the user is experiencing difficulties with their eyesight. People whose eyesight is deteriorating will often begin to squint more than they normally do. The system10can be configured to detect the user squinting, for example, by detecting wrinkles that appear on the user's face near their eyes when they squint. The system10may also detect that the user is squinting in other ways. Once the application is launched, the system10can perform a variety of eyesight tests for the user, for example by displaying a series of numbers and/or letters that get increasingly small, and having the user identify which series they can still see properly.

In some implementations, the system10can execute a stretching application. As a part of this application, the system10can detect the user's posture (whether sitting, standing, walking, running, jumping, etc.) and recommend exercises to improve the user's posture.

FIGS.24A-24Dshow an augmented reality application the system10can execute, similar to that shown inFIG.22C. The augmented reality application can cause the display14to show to the user various modified images or videos of themselves. For example, the user may be holding a beauty product that is capable of being applied to a person (e.g., lipstick, eye shadow, hair dye, etc.). The display14can continuously display a real-time video feed of the user (or a portion of the user) that shows the user as if they had applied the beauty product to themselves.FIG.24Ashows the system10in a standby mode. In the standby mode, no images or videos of the user or the surrounding area are shown in the display14. Various other icons may be shown on the display14however. In the standby mode, any objects in front of the mirror12would generally be visible as a reflection in the mirror12, as the display14generally does not display anything when the system10is in the standby mode.

FIG.24Billustrates a view of the system10once a user has walked near the system10in front of the mirror12. In this view, the display14is continuously showing a real-time video feed302of a user304. However, the display14could also be off such that the user304simply sees his or her reflection in the mirror12. In some implementations, the system10constantly monitors the area surrounding the mirror12using the camera20, such that the display14automatically displays the video feed302of the user304once the user304walks in front of the mirror12, or after a certain time period following the user304walking in front of the mirror12. In other implementations, the user304must activate the system10or wake the system10from the standby mode in order to see the video feed302of themselves in the display14. In either implementation, the user304may view their reflection in the mirror12prior to the display14activating and the display14showing the real-time video feed302of the user304.

FIG.24Cshows the user304holding up an object306in the field of view of the camera20. The processor22of the system10runs the object recognition (OR) algorithm, either continuously, or upon some sort of command from the user (e.g., a voice command, a touch command, a gesture command, etc.), in order to detect and recognize the object306that the user304is holding. Similar toFIGS.17A and17B, the system10can recognize the object306using the data generated by the camera20. The OR algorithm can also detect specific characteristics of the object306. For example, if the OR algorithm detects that the user304is holding a tube of lipstick, the OR algorithm can identify the brand and color of the lipstick that the user304is holding. The OR algorithm can also be used to detect other types of objects306that the user304may be holding, as described herein.

Once the system10has identified the object that the user304is holding, the display14can modify the continuously displayed real-time video feed302of the user304based on the identified object306.FIG.24Dshows a view of the system10where the system10has modified the continuously displayed real-time video feed302of the user304ofFIG.24C. InFIG.24C, the video feed302on the display14shows the user304and the portion of the area surrounding the user304. In the modified video feed inFIG.24D, both a modified portion of the user304and an unmodified portion of the user304are shown. The modified portion appears as if the user304applied the lipstick to their lips. The unmodified portion of the user304appears normal. Thus, the system10modifies the continuously display real-time video feed302of the user304such that the video feed302of the user304shows the user304as if they used the object that the OR algorithm detected they were holding. For example, if the object306is a beauty product that can be applied to the user304, the modified video feed302of the user304will show the user304as if they had applied the beauty product to themselves. The display14can display an indication to the user304of what the object is. In the implementation illustrated inFIG.24C, the display14displays the word “Lipstick.”

In some implementations, the modified video feed302of the user304shows both the modified portion of the user304and the unmodified portion of the user304. The user304's entire face and head are thus shown on the display14. In this implementation, the light emitted from the display14through the mirror12toward the user304is stronger than the light reflected off of the mirror12back toward the user304, and thus the user304sees the both the modified portion and the unmodified portion of themselves in the real-time video feed302on the display14. In other implementations, the display14only shows the modified portion of the user304, while the unmodified portion of the user304(e.g., the rest of the user304's face and/or head) is shown as a reflection in the mirror12. For example, if the system10shows the user304what the user304would look like if they applied lipstick, the display14may only show the user304's lips colored the same color as the lipstick, with the rest of the display14being dark. The display14will show the user304's lips at a location that is generally aligned on the mirror12with the user304's actual lips. The user304will thus see the modified version of their lips in the display14at a location aligned with where they would normally instead see the reflection of their un-augmented lips. Thus, the user304will see (1) the modified portion of themselves (e.g., their lips) on the real-time video feed302on the display14through the mirror12, and (2) the unmodified portion of themselves (e.g., their face and head except for their lips) as a reflection in the mirror12.

The system10can show real-time video feeds302of the user304that are modified in a variety of ways. For example, the modified video feed302of the user304can show: (1) the user304wearing glasses, (2) the user304not wearing glasses if the user304normally does wear glasses, (3) the user304with a different hair color, (4) the user304with a different hair style, (5) the user304with their face shape retouched or altered, (6) the user304with abnormalities on their skin (such as freckles or acne) removed, (7) the user304with a modified complexion (e.g. with a tan or a sunburn), or (8) any other suitable or desired modification.

In some implementations, the modified video feed302is displayed automatically upon detecting the object306the user304is holding. In other implementations, the modified video feed302is displayed responsive to the system10receiving or identifying some type of prompt, such as a user-selection of a displayed user-selectable icon, a voice command, or user-actuation of a physical trigger mechanism (such as a button or touch-sensitive controller). The prompt could also be bringing the object306into a field of view of the camera20so that the system10can detect and identify the object306. In some implementations, the ambient conditions in the room where the user304is located (such as light, temperature, humidity, etc.), the local weather conditions, or a change in the ambient conditions or local weather conditions can also serve as the prompt to the system10to modify the real-time video feed302of the user304. Moreover, the actual modification shown by the modified video feed302can be based on something other than the identification of an object306being held by the user304. For example, the prompt received or identified by the system10that causes the display14to display the modified video may also select what type of modification to show in the modified video. The modification in the modified video can also be based on the ambient conditions in the room where the user304is located, the local weather conditions, or any other suitable mechanism.

Generally, the system10must be able to generate image data of the user, process the image data, and modify the continuously displayed video feed of the user in real time or near real time. This time period is known as the system10's response time. Generally, the system10can process the image data and modify the video feed in about 20 milliseconds. Thus, for example, when the user304moves within the field of view of the camera20, the user304's new location is shown on the display14within about 20 milliseconds or less. In other implementations, the response time of the system10may be less than about 20 milliseconds, between about 5 milliseconds and 20 milliseconds, between about 10 milliseconds and about 20 milliseconds, or less than about 10 milliseconds. In other implementations, the system10can process the image data and modify the video feed of the user304within about 50 microseconds. between about 10 microseconds and about 50 microseconds, between about 20 microseconds and about 40 microseconds, between about 10 microseconds and about 30 microseconds, or about 20 microseconds. In still other implementations, the system10processes the image data and modifies the video feed within about 1 second or less. As used herein, the term “real-time” as used at least in connection with the continuously displayed video feed of the user, refers to implementations where the system10processes image data and modifies the video feed of the user within the time frames or periods discussed herein.

In implementations where the display14only shows the modified portion of the user304in the real-time video feed302, the response time of the system10can be reduced. Because the processor(s)22of the system10need only to process image data corresponding to the portion of the user304that is being modified in the video feed302, the modification of the video feed302generally takes less time than if a video feed302of all of the user304(or even a larger portion of the user304) was being modified.

When processing the image data generated by the camera20, the system10can flip the resulting image or video feed302of the user304about a vertical axis so that the resulting image or video feed302matches what the user304will see in the mirror12. Because the user304's reflection in the mirror12is flipped about a vertical axis, a normal image or video feed captured by a camera20and displayed on a display14would not match the user304's reflection. Thus, the image or video feed302that is displayed on the display14is flipped about a vertical axis to match the reflection in the mirror12.

The system10can also be used to determine the mood of the user304and take actions to alter or improve the user304's mood. In one implementation, the system10can determine if the user304has a negative mood by detecting certain actions or characteristics of the user304. For example, the system10could detect when the user304is frowning, when the user304is slouching their shoulders, when the user304is hanging their head, or any other suitable action that may indicate a negative mood. After detecting a negative mood, the system10can take a number of actions to try to improve the user304's mood. For example, the system10could play a favorite song of the user304, improve the ambient lighting in the area surrounding the user304, show the user304a stored photo or video of a positive memory, show the user304desirable local weather conditions (current or future), show a modified video of the user304with some desirable feature, or any other suitable action.

In another implementation, the system10can determine if the user304has a positive mood, for example by detecting the user304smiling, or by detecting any other actions associated with a positive mood. The system10can store information about why the user304is in a positive mood, which can be used in the future to try and improve the user304's negative mood.

In still other implementations, the system10can act as an advisor or motivator after detecting various characteristics or features of the user304. For example, if the system10knows or detects that the user304has gained weight, the system10could suggest to the user304that they exercise more frequently, take more daily steps, eat healthier, etc. In another example, the system10could show the user304a modified video or photo of themselves either in shape or out of shape, in an attempt to motivate the user304. In a further example, the system10may detect if the user304is not getting enough sleep by monitoring for dark circles under the user304's eyes, excessive yawning, etc., and suggest to the user304that they get more sleep. In yet another example, the system10may detect if the user304is having difficulty seeing, such as by detecting that the user304is squinting or opening their eyes excessively wide more than usual. The system10could administer an eye test, and/or suggest that the user304visit an eye doctor to obtain glasses or contacts.

While the present disclosure has been described with reference to one or more particular implementations, those skilled in the art will recognize that many changes may be made thereto without departing from the spirit and scope of the present disclosure. Each of these implementations and obvious variations thereof is contemplated as falling within the spirit and scope of the present disclosure. It is also contemplated that additional implementations according to aspects of the present disclosure may combine any number of features from any of the implementations described herein.

ALTERNATIVE IMPLEMENTATIONS

Alternative Implementation 1. A system comprising: a mirror having a first side and an opposing second side, the mirror being configured to permit a first portion of light incident on the first side to transmit therethrough and to permit a second portion of the light incident on the first side to reflect therefrom; an electronic display positioned adjacent to the second side of the mirror such that the electronic display is at least partially visible through the first side of the mirror responsive to the electronic display being at least partially activated; one or more sensors positioned generally about a periphery of the mirror and being configured to detect a presence of and a relative location of an object positioned adjacent to the first side of the mirror; a light source configured to produce light and at least partially aid in illuminating the object responsive to the object being adjacent to the first side of the mirror; and a camera positioned adjacent the second side of the mirror, the camera being configured to detect the object.

Alternative Implementation 2. The system of Alternative Implementation 1, further comprising one or more processors configured to adjust a color or an intensity of the light source.

Alternative Implementation 3. The system of Alternative Implementation 1, wherein the light source includes one more light-emitting diodes.

Alternative Implementation 4. The system of Alternative Implementation 1, wherein the mirror has a first axis and a second axis perpendicular to the first axis, and wherein the one or more sensors are configured to detect a first axial position of the object relative to the first axis of the mirror and a second axial position of the object relative to the second axis of the mirror.

Alternative Implementation 5. The system of Alternative Implementation 4, wherein the one or more sensors includes a first set of one or more transmitters disposed along a first portion of the mirror parallel to the first axis and a first set of one or more receivers disposed along a second opposing portion of the mirror parallel to the first axis, the first set of one or more transmitters and the first set of one or more receivers configured to detect the first axial position of the object.

Alternative Implementation 6. The system of Alternative Implementation 5, wherein the one or more sensors includes a second set of one or more transmitters disposed along a third portion of the mirror parallel to the second axis and a second set of one or more receivers disposed along a fourth opposing portion of the mirror parallel to the second axis, the second set of one or more transmitters and the second set of one or more receivers configured to detect the second axial position of the object.

Alternative Implementation 7. The system of Alternative Implementation 6, wherein the first axis is a vertical axis and the second axis is a horizontal axis.

Alternative Implementation 8. The system of Alternative Implementation 6, wherein at least a portion of the one or more sensors is disposed within a sensor frame that is coupled to the periphery of the mirror.

Alternative Implementation 9. The system of Alternative Implementation 1, further comprising one or more processors configured to cause the display to aid in illuminating the object by activating at least a portion of the display.

Alternative Implementation 10. The system of Alternative Implementation 9, wherein the one or more processors are configured to activate the portion of the display responsive to at least one of the one or more sensors detecting the presence and relative location of the object positioned adjacent to the first side of the mirror.

Alternative Implementation 11. The system of Alternative Implementation 9, wherein the activated portion of the display includes a ring-shaped configuration such that the activated portion of the display is generally aligned with an outer periphery of the object.

Alternative Implementation 12. The system of Alternative Implementation 11, wherein the activated portion of the display substantially encloses a non-activated portion of the display, the non-activated portion of the display being generally aligned with an interior of the outer periphery of the object.

Alternative Implementation 13. The system of Alternative Implementation 9, wherein the activated portion of the display has a shape that corresponds to a shape of the object.

Alternative Implementation 14. The system of Alternative Implementation 9, wherein the object is a face, and wherein the display is caused to be activated responsive to the face being located within a threshold distance from the mirror.

Alternative Implementation 15. The system of Alternative Implementation 14, wherein the threshold distance is less than about five inches.

Alternative Implementation 16 The system of Alternative Implementation 1, further comprising an ambient light sensor configured to detect an ambient light condition.

Alternative Implementation 17. The system of Alternative Implementation 16, further comprising one or more processors configured to determine a difference between the detected ambient light condition and a desired light condition.

Alternative Implementation 18. The system of Alternative Implementation 17, wherein at least one of the one or more processors is configured to adjust the light produced by the light source based on the determined difference between the detected ambient light condition and the desired light condition, such that the ambient light condition and the light produced by the light source combine to cause the object to be illuminated according to the desired light condition.

Alternative Implementation 19. The system of Alternative Implementation 16, wherein the ambient light sensor is included in the camera.

Alternative Implementation 20. The system of Alternative Implementation 1, wherein the display is configured to display one or more user-selectable icons visible to a user through the mirror, and wherein one or more processors are configured to adjust the light produced by the light source responsive to a selection of one of the one or more user-selectable icons.

Alternative Implementation 21. The system of Alternative Implementation 1, wherein the display is configured to display real-time video images captured by the camera.

Alternative Implementation 22. The system of Alternative Implementation 1, further comprising one or more processors configured to execute a first application, the display being configured to display a first type of information related the first application.

Alternative Implementation 23. The system of Alternative Implementation 22, wherein the camera is configured to monitor an area adjacent to the first side of the mirror, and wherein at least one of the one or more processors is configured to identify the object.

Alternative Implementation 24. The system of Alternative Implementation 23, wherein responsive to the identification of the object, at least one of the one or more processors is configured to cause the display to display a second type of information related to the first application, the second type of information being (i) different from the first type of information and (ii) based on the identified object.

Alternative Implementation 25. The system of Alternative Implementation 23, wherein responsive to the identification of the object, at least one of the one or more processors is configured to execute a second application different from the first application, the second application being based on the identified object.

Alternative Implementation 26. The system of Alternative Implementation 1, wherein the object is a toothbrush, a razor, a comb, a makeup implement, a face of a user, or any combination thereof.

Alternative Implementation 27. A method of modifying execution an application, comprising: providing an electronic display device and a mirror, the mirror having a first side and an opposing second side, the electronic display device being positioned adjacent to the second side of the mirror such the electronic display device is at least partially visible through the first side of the mirror responsive to the electronic display device being activated; executing, on one or more processing devices communicatively coupled to the electronic display device, a first application; displaying, on the electronic display device, a first type of information related to the first application; monitoring an area adjacent to the first side of the mirror with a camera communicatively coupled to the one or more processing devices; detecting, with the one or more processing devices, an object disposed in the area adjacent to the first side of the mirror; identifying, with the one or more processing devices, the detected object from a plurality of pre-determined potential objects; and responsive to identifying the detected object, modifying the execution of the first application based on the identified detected object.

Alternative Implementation 28. The method of Alternative Implementation 27, wherein the modifying the execution of the first application includes displaying a second type of information related to the first application, the second type of information being different from the first type of information.

Alternative Implementation 29. The method of Alternative Implementation 27, wherein the modifying the execution of the first application includes executing a second application on the processing device different from the first application.

Alternative Implementation 30. The method of Alternative Implementation 27, wherein the object is a toothbrush, a razor, a makeup implement, or any combination thereof.

Alternative Implementation 31. A method of illuminating a face, comprising: providing an electronic display device and a mirror, the mirror having a first side and an opposing second side, the electronic display device being positioned adjacent to the second side of the mirror such the electronic display device is at least partially visible through the first side of the mirror responsive to the electronic display device being activated; monitoring an area adjacent to the first side of the mirror with one or more sensors; detecting, with the one or more sensors, the face responsive to the face being positioned within a threshold distance from a surface of the mirror on the second side of the mirror; determining, using one or more processors communicatively coupled to the one or more sensors, an outer periphery of the face; and activating a portion of the electronic display device to illuminate the face.

Alternative Implementation 32. The method of Alternative Implementation 31, wherein the activated portion of the electronic display device includes a ring-shaped configuration such that the activated portion of the electronic display device is generally aligned with the outer periphery of the face.

Alternative Implementation 33. The method of Alternative Implementation 32, wherein the activated portion of the electronic display device substantially encloses a non-activated portion of the display, the non-activated portion of the electronic display device being generally aligned with an interior of the outer periphery of the face.

Alternative Implementation 34. The method of Alternative Implementation 31, wherein the activated portion of the electronic display device has a shape that generally corresponds to a shape of the face.

Alternative Implementation 35. The method of Alternative Implementation 31, wherein the threshold distance is less than about five inches.

Alternative Implementation 36. A system for illuminating a face of a user, the system comprising: a smart mirror device including: a frame; a mirror coupled to the frame; one or more sensors coupled to the frame and positioned generally adjacent to at least a portion of a periphery of the mirror; an electronic display device coupled to the frame such that the electronic display device is positioned adjacent to a portion of the mirror; and one or more processors configured to execute instructions stored in one or more memory devices such that the smart mirror device is caused to: responsive to the face of the user being positioned within a threshold distance from a surface of the mirror, detect, via at least one of the one or more sensors, the face of the user; determine an outer periphery of the face; and activate a portion of the electronic display device to aid in illuminating the face.

Alternative Implementation 37. The system of Alternative Implementation 36, wherein the activated portion of the electronic display has a generally oval shape, a generally circular shape, a generally curved shape, or any combination thereof.

Alternative Implementation 38. The system of Alternative Implementation 36, wherein the activated portion of the electronic display also aids in illuminating a reflection of the face in the mirror.

Alternative Implementation 39. The system of Alternative Implementation 36, wherein the mirror is coupled to the frame such that the mirror is less than one inch from a screen of the electronic display.

Alternative Implementation 40. A method of compensating for an ambient light condition, comprising: providing an electronic display device and a mirror, the mirror having a first side and an opposing second side, the electronic display device being positioned adjacent to the second side of the mirror such the electronic display device is at least partially visible through the first side of the mirror responsive to the electronic display device being activated; detecting, using an ambient light sensor, the ambient light condition illuminating an object; determining, using one or more processors communicatively coupled to the ambient light sensor, a difference between the detected ambient light condition and a desired light condition; producing light using one or more lighting sources based on the determined difference between the ambient light condition and the desired light condition; and illuminating the object with the produced light such that the ambient light condition and the produced light combine to cause the object to be illuminated according to the desired lighting condition.

Alternative Implementation 41. The method of Alternative Implementation 40, wherein the ambient light sensor is a camera.

Alternative Implementation 42. The method of Alternative Implementation 41, wherein the ambient light condition is detected by capturing an image of an object and analyzing the captured image.

Alternative Implementation 43. A system for compensating for an ambient lighting condition, the system comprising: a smart mirror device including: a frame; a mirror coupled to the frame; an ambient lighting sensor coupled to the frame; one or more lighting sources; an electronic display device coupled to the frame such that the electronic display device is positioned adjacent to a portion of the mirror; and one or more processors configured to execute instructions stored in one or more memory devices such that the smart mirror device is caused to: detect, using the ambient lighting sensor, the ambient lighting condition adjacent to the system; determine a difference between the detected ambient lighting condition and a desired lighting condition; and emit light, via one or more lighting sources, based on the determined difference between the ambient lighting condition and the desired lighting condition.

Alternative Implementation 44. A method of capturing an image of a user, comprising: providing an electronic display device and a mirror, the mirror having a first side and an opposing second side, the electronic display device being positioned adjacent to the second side of the mirror such the electronic display device is at least partially visible through the first side of the mirror responsive to the electronic display device being activated; illuminating, using light produced by one or more light sources, a user located adjacent to the first side of the mirror; capturing, using a camera, a first image of the user located adjacent to the first side of the mirror; displaying on the electronic display device the captured first image of the user; displaying, on the electronic display device, one or more user-selectable icons, each icon corresponding to a different lighting condition; detecting a user selection of at least one of the one or more user-selectable icons; adjusting the light produced by the one or more light sources based on the user selection of the at least one of the one or more user-selectable icons such that the user is illuminated by a selected lighting condition; and capturing, using the camera, a second image of the user illuminated by the selected lighting condition.

Alternative Implementation 45. The system of Alternative Implementation 6, wherein the mirror and the sensor frame have an identical shape.

Alternative Implementation 46. The system of Alternative Implementation 6, wherein the mirror and the sensor frame have a different shape.

Alternative Implementation 47. The system of Alternative Implementation 6, wherein the sensor frame has a rectangular shape, a square shape, a triangle shape, a circle shape, or an oval shape.

Alternative Implementation 48. The system of Alternative Implementation 6, wherein the sensor frame has a rectangular shape, a square shape, a triangle shape, a circle shape, or an oval shape.

Alternative Implementation 49. A system comprising: a mirror having a first side and an opposing second side, the mirror being configured to permit a first portion of light incident on the first side to transmit therethrough and to permit a second portion of the light incident on the first side to reflect therefrom; a first electronic display positioned adjacent to the second side of the mirror such that the first electronic display is at least partially visible through the first side of the mirror responsive to the first electronic display being at least partially activated; a second electronic display positioned adjacent to the second side of the mirror such that the second electronic display is at least partially visible through the first side of the mirror responsive to the second electronic display being at least partially activated, the second electronic display being spaced apart from the first electronic display such that a gap is defined between the first electronic display and the second electronic display; one or more sensors positioned generally about a periphery of the mirror and being configured to detect a presence of and a relative location of an object positioned adjacent to the first side of the mirror; a light source configured to produce light and at least partially aid in illuminating the object responsive to the object being adjacent to the first side of the mirror; and a camera configured to detect the object, the camera being positioned adjacent the second side of the mirror and positioned within the gap between the first electronic display and the second electronic display.

Alternative Implementation 50. A system comprising: a frame; a mirror coupled to the frame, the mirror having a first side and an opposing second side, the mirror being configured to permit a first portion of light incident on the first side to transmit therethrough and to permit a second portion of the light incident on the first side to reflect therefrom; an electronic display couple to the frame such that the electronic display is positioned adjacent to the second side of the mirror; a plurality of sensors positioned generally about at least a portion of a periphery of the mirror and being configured to detect a presence of and a relative location of an object positioned adjacent to the first side of the mirror; a camera positioned adjacent the second side of the mirror and positioned such that the object is within a field of view of the camera responsive to the object being positioned adjacent to the first side of the mirror; and an obstruction coupled the frame and being configured move between a first position where the obstruction obscures the field of view of the camera and a second position where the obstruction does not obscure the field of view of the camera.

Alternative Implementation 51. The system of Alternative Implementation 50, wherein the obstruction is pivotally coupled to the frame.

Alternative Implementation 52. The system of Alternative Implementation 50, wherein the obstruction is slidably coupled to the frame.

Alternative Implementation 53. The system of Alternative Implementation 50, wherein the obstruction includes an opaque material.

Alternative Implementation 54. The system of Alternative Implementation 50, wherein the obstruction includes a partially transparent material such that only a portion of light is able to pass therethrough.

Alternative Implementation 55. The system of Alternative Implementation 50, wherein the obstruction includes one or more optical filters.

Alternative Implementation 56. The system of Alternative Implementation 50, wherein the obstruction is biased towards the first position.

Alternative Implementation 57. The system of Alternative Implementation 50, wherein the obstruction is biased towards the second position.

Alternative Implementation 58. The system of Alternative Implementation 50, wherein the obstruction is configured to be biased towards the first position or towards the second position.

Alternative Implementation 59. The system of Alternative Implementation 50, wherein the obstruction is configured to be manually moved between the first position and the second position.

Alternative Implementation 60. The system of Alternative Implementation 50, wherein the obstruction is configured to be automatically moved between the first position and the second position using one or more electronic actuation devices.

Alternative Implementation 61. The system of Alternative Implementation 60, wherein the one or more activation devices include a solenoid, a spring, a lever, a magnet, or any combination thereof.

Alternative Implementation 62. The system of Alternative Implementation 50, wherein the electronic display is at least partially visible through the first side of the mirror only when to the electronic display is activated.

Alternative Implementation 63. The system of Alternative Implementation 50, further comprising a light source configured to produce light and at least partially aid in illuminating the object.

Alternative Implementation 64. A system comprising: a frame; a mirror coupled to the frame; an electronic display couple to the frame such that the electronic display is positioned adjacent to a rear side of the mirror; a camera coupled to the frame such that a field of view of the camera includes an area adjacent to a front side of the mirror; and an obstruction coupled the frame and being configured move between a first position and a second position, responsive to the obstruction being in the first position, the obstruction is configured to obscure at least a portion of the field of view of the camera, responsive to the obstruction being in the second position, the obstruction is configured to not obscure the field of view of the camera.

Alternative Implementation 65. The system of Alternative Implementation 64, wherein the mirror is a two-way mirror such that the field of view of the camera is able to extend into the area adjacent to the front side of the mirror.

Alternative Implementation 66. The system of Alternative Implementation 64, wherein the electronic display device includes an aperture therethrough such that the field of view of the camera is able to extend through the aperture and into the area adjacent to the front side of the mirror.

Alternative Implementation 67. The system of Alternative Implementation 66, wherein the aperture is located in a generally central portion of the electronic display device.

Alternative Implementation 68. A system for providing a tutorial, the system comprising: a smart mirror device including: a frame; a mirror coupled to the frame; an electronic display device coupled to the frame such that the electronic display device is positioned adjacent to a portion of the mirror; and one or more processors configured to execute instructions stored in one or more memory devices such that the smart mirror device is caused to: responsive to receiving an input to begin a first tutorial, display, on the electronic display device, (i) a first looping video illustrating a human performing a how-to tutorial for a first type of activity, (ii) a second looping video illustrating an animation associated with the how-to tutorial shown in the first looping video, and (iii) a real-time video feed of a user; and responsive to receiving a magnifier input associated with a portion of the user in the displayed real-time video feed, display, on the electronic display device, a zoomed in view of the user including the portion of the user.

Alternative Implementation 69. The system of Alternative Implementation 68, wherein the smart mirror further includes a light source that is configured to emit light onto a portion of the user.

Alternative Implementation 70. The system of Alternative Implementation 69, wherein the light source includes one or more lasers that are configured to project one or more shapes on the user.

Alternative Implementation 71. The system of Alternative Implementation 70, wherein the one or more shapes include dots, circles, squares, triangles, curves, arcs, arches, lines, or any combination thereof

Alternative Implementation 72. The system of Alternative Implementation 70, wherein the one or more shapes projected onto the user indicate one or more locations for the user to apply one or more products associated with the first type of activity.

Alternative Implementation 73. The system of Alternative Implementation 68, wherein the first type of activity is application of makeup, application of moisturizer, application of hair dye, application of hair gel, application of shaving cream, shaving, brushing teeth, eyebrow waxing, waxing, eyebrow threading, threading, facials, ear cleaning, hair styling, application of contact lenses, application of facial masks, or any combination thereof.

Alternative Implementation 74. The system of Alternative Implementation 68, wherein the one or more processors are further configured to execute instructions stored in the one or more memory devices such that the smart mirror device is caused to: display, adjacent to the real-time video feed of the user, a selectable magnifier element.

Alternative Implementation 75. A system for providing a tutorial, the system comprising: a frame; a mirror coupled to the frame; an electronic display device coupled to the frame such that the electronic display device is positioned adjacent to a portion of the mirror; and one or more processors configured to execute instructions stored in one or more memory devices such that the system is caused to: responsive to receiving an input to begin a first tutorial, display, on the electronic display device, a first looping video illustrating a human performing a how-to tutorial for a first type of activity.

Alternative Implementation 76. The system of Alternative Implementation 75, wherein the system is further caused to display a second looping video illustrating an animation associated with the how-to tutorial shown in the first looping video.

Alternative Implementation 77. The system of Alternative Implementation 76, wherein the second looping video is displayed below the first looping video and aligned therewith such that a centerline of the first looping video and a centerline of the second looping video are coincident.

Alternative Implementation 78. The system of Alternative Implementation 75, wherein the system is further caused to display a real-time video feed of a user.

Alternative Implementation 79. The system of Alternative Implementation 78, wherein the system is further caused to, responsive to receiving a magnifier input associated with a portion of the user in the displayed real-time video feed, display, on the electronic display device, a zoomed in view of the user including the portion of the user.

Alternative Implementation 80. The system of Alternative Implementation 75, wherein the system is further caused to display a still image of a user.

Alternative Implementation 81. The system of Alternative Implementation 80, wherein the system is further caused to, responsive to receiving a magnifier input associated with a portion of the user in the displayed still image, display, on the electronic display device, a zoomed in still image of the user including the portion of the user.

Alternative Implementation 82. A system for providing a tutorial, the system comprising: a frame; a mirror coupled to the frame; an electronic display device coupled to the frame such that the electronic display device is positioned adjacent to a portion of the mirror; and one or more processors configured to execute instructions stored in one or more memory devices such that the system is caused to: responsive to receiving an input to begin a first tutorial, display, on the electronic display device, (i) a first looping video illustrating a human performing a how-to tutorial for a first type of activity and (ii) a real-time video feed of a user; and modify the real-time video feed of the user to overlay on a portion of the user one or more template features associated with the first type of activity.

Alternative Implementation 83. The system of Alternative Implementation 82, wherein the first type of activity is application of makeup, application of moisturizer, application of hair dye, application of hair gel, application of shaving cream, shaving, brushing teeth, eyebrow waxing, waxing, eyebrow threading, threading, facials, ear cleaning, hair styling, application of contact lenses, application of facial masks, or any combination thereof.

Alternative Implementation 84. The system of Alternative Implementation 82, wherein the one or more template features include one or more dots, one or more circles, one or more squares, one or more triangles, one or more curves, one or more arcs, one or more arches, one or more lines, or any combination thereof.

Alternative Implementation 85. The system of Alternative Implementation 82, wherein the one or more template features aid the user in conducting the first type of activity.

Alternative Implementation 86. A system for capturing and displaying images of a user, the system comprising: a frame; a mirror coupled to the frame; an electronic display device coupled to the frame such that the electronic display device is positioned adjacent to a portion of the mirror; a camera coupled to the frame; and one or more processors configured to execute instructions stored in one or more memory devices such that the system is caused to: generate, using the camera, video data associated with at least a portion of the user; display, on the electronic display device, the generated video data as a real-time video feed of at least a portion of the user; capture, from the video data, first image data reproducible as a first image of the at least a portion of the user; display the first image and a first set of selectable options, the first set of selectable options being associated with a first characteristic of the user; receive, via an input device, a selection of one of the first set of selectable options; and responsive to receiving the selection of the one or the first set of selectable options, display a second image of the user, the second image of the user being a modified version of the first image of the user and illustrating a proposed modification to the first characteristic of the user, the modification being based on the selection of the one of the first set of selectable options.

Alternative Implementation 87. The system of Alternative Implementation 86, wherein the system is further caused to: display a second set of selectable options, the second set of selectable options being associated with a second characteristic of the user; and receive, via the input device, a selection of one of the second set of selectable options.

Alternative Implementation 88. The system of Alternative Implementation 87, wherein the system is further caused to, responsive to receiving the selection of one of the second set of selectable options second, display a third image of the user, the third image of the user being a second modified version of the first image of the user, the second modification being based on the selection of the one of the second set of selectable options.

Alternative Implementation 89. The system of Alternative Implementation 88, wherein the system is further configured to simultaneously display the first image of the user, the second image of the user, and the third image of the user.

Alternative Implementation 90. The system of Alternative Implementation 87, wherein the system is further configured to, responsive to receiving the selection of the one of the second set of selectable options, display (i) the first image of the user, (ii) the second image of the user; and (iii) a third image associated with the selection of the one of the second set of selectable options, the third image illustrating a proposed modification to the second characteristic of the user.

Alternative Implementation 91. The system of Alternative Implementation 87, wherein the first characteristic of the user is a hair color of the user.

Alternative Implementation 92. The system of Alternative Implementation 91, wherein the first image of the user shows the user with an existing hair color and wherein the second image of the user shows the user with a modified hair color, the modified hair color corresponding to the selection of the one of the first set of selectable options.

Alternative Implementation 93. The system of Alternative Implementation 91, wherein each of the first set of selectable options is associated with a distinct hair color.

Alternative Implementation 94. The system of Alternative Implementation 91, wherein the second characteristic of the user is a hair style of the user.

Alternative Implementation 95. The system of Alternative Implementation 94, wherein each selectable option of the second set of selectable options is associated with a distinct hair style.

Alternative Implementation 96. The system of Alternative Implementation 86, wherein each of the first set of selectable options is a unique proposed modification of the first characteristic of the user.

Alternative Implementation 97. The system of Alternative Implementation 86, wherein each of the first set of user selectable options includes an image associated with a unique proposed modification of the first characteristic of the user.

Alternative Implementation 98. A system for displaying images, the system comprising: a frame; a mirror coupled to the frame; an electronic display device coupled to the frame such that the electronic display device is positioned adjacent to a portion of the mirror; a camera coupled to the frame; and one or more processors configured to execute instructions stored in one or more memory devices such that the system is caused to: display, on the electronic display device, a real-time video feed of a user; capture first image data reproducible as a first image of the user; identify an object being held by the user in the first image; and responsive to identifying the object being held by the user in the first image, display a second image of the user, the second image of the user being a modified version of the first image of the user, the modification being based on the identified object being held by the user.

Alternative Implementation 99. The method of Alternative Implementation 98, wherein the object includes hair dye and wherein the second image of the user illustrates the user having a hair color associated with the hair dye.

Alternative Implementation 100. The method of Alternative Implementation 98, wherein the object includes lipstick and wherein the second image of the user illustrates the user having a lip color associated with the lipstick.

Alternative Implementation 101. The method of Alternative Implementation 98, wherein the object includes makeup and wherein the second image of the user is a modified version of the first image that is modified based on the identified makeup.

Alternative Implementation 102. A system for displaying images, the system comprising: a frame; a mirror coupled to the frame; an electronic display device coupled to the frame such that the electronic display device is positioned adjacent to a portion of the mirror; a camera coupled to the frame; and one or more processors configured to execute instructions stored in one or more memory devices such that the system is caused to: display, on the electronic display device, a real-time video feed of a user; receive, via an input device, a selection of an area of interest of the user shown in the real-time video feed of the user; display, on the electronic display device, a zoomed-in real-time video feed of the area of interest of the user; and continue to display, on the electronic display device, the zoomed-in real time video feed of the area of interest of the user responsive to movement of the user.

Alternative Implementation 103. A system comprising: a frame; a mirror coupled to the frame; an electronic display device coupled to the frame such that the electronic display device is positioned adjacent to at least a portion of the mirror; a camera coupled to the frame, the camera having a field of view; and one or more processors configured to execute instructions stored in one or more memory devices such that the system is caused to: continuously display, on the electronic display device, a real-time video feed of at least a portion of a user positioned within the field of view of the camera; identify an object within the field of view of the camera; and subsequent to the identification, modify the continuously displayed real-time video feed based on at least the identified object.

Alternative Implementation 104. The system of Alternative Implementation 103, wherein the frame is configured to be mounted to a wall.

Alternative Implementation 105. The system of Alternative Implementation 103, wherein the at least a portion of the user includes a face of the user, hair of the user, a torso of the user, an arm of the user, a leg of the user, a head of the user, or any combination thereof.

Alternative Implementation 106. The system of Alternative Implementation 103, wherein the identified object is a beauty product capable of being applied to the user, and wherein the modified continuously displayed real-time video feed shows a result of the beauty product being applied to the user.

Alternative Implementation 107. The system of Alternative Implementation 106, wherein the identified object includes hair dye, and wherein the modified continuously displayed real-time video feed shows the user having a hair color associated with the hair dye.

Alternative Implementation 108. The system of Alternative Implementation 106, wherein the identified object includes lipstick, and wherein the modified continuously displayed real-time video feed shows the user having a lip color associated with the lipstick.

Alternative Implementation 109. The system of Alternative Implementation 106, wherein the identified object includes makeup, and wherein modified continuously displayed real-time video feed shows the user with the makeup applied to the user.

Alternative Implementation 110. The system of Alternative Implementation 103, wherein the identified object is a pair of glasses, and wherein the second real-time video of the user shows the user wearing the pair of glasses.

Alternative Implementation 111. The system of Alternative Implementation 103, wherein the identified object is held by the user.

Alternative Implementation 112. The system of Alternative Implementation 103, wherein the continuously displayed real-time video feed is modified responsive to the system receiving a prompt from the user.

Alternative Implementation 113. The system of Alternative Implementation 112, wherein the prompt from the user is the identification of the object within the field of view of the camera, a user selection of a user-selectable icon displayed on the display, a voice command, or a user gesture.

Alternative Implementation 114. The system of Alternative Implementation 103, wherein the modified continuously displayed real-time video feed of the at least a portion of the user shows only a modified portion of the user, such that (i) the modified portion of the user is visible to the user in the display through the mirror and (ii) an unmodified portion of the user is visible to the user in the mirror.

Alternative Implementation 115. The system of Alternative Implementation 103, wherein the modified continuously displayed real-time video feed of at the least a portion of the user shows a modified portion of the user and an unmodified portion of the user, such that neither the modified portion of the user or the unmodified portion of the user are visible in the display.

Alternative Implementation 116. The system of Alternative Implementation 103, wherein the continuously displayed real-time video is modified within about 20 milliseconds of the identification of the object.

Alternative Implementation 117. The system of Alternative Implementation 103, wherein the continuously displayed real-time video is modified within about 20 microseconds of the identification of the object.

Alternative Implementation 118. A system, the system comprising: a frame; a mirror coupled to the frame; an electronic display device coupled to the frame such that the electronic display device is positioned adjacent to a portion of the mirror; a camera coupled to the frame, the camera having a field of view; and one or more processors configured to execute instructions stored in one or more memory devices such that the system is caused to: continuously display, on the electronic display device, a real-time video feed of at least a portion of a user positioned within the field of view of the camera; and subsequent to receiving a prompt, modify the continuously displayed real-time video feed based on at least the prompt.

Alternative Implementation 119. The system of Alternative Implementation 118, wherein the prompt is an identification of an object within the field of view of the camera, a user-selection of a user-selectable icon displayed on the display, a voice command, or a user gesture.

Alternative Implementation 120. The system of Alternative Implementation 118, wherein the modification is based on an identification of an object within the field of view of the camera, a user-selection of a user-selectable icon displayed on the display, a voice command, a user gesture, ambient conditions, or local weather conditions.

Alternative Implementation 121. The system of Alternative Implementation 118, wherein the modified continuously displayed real-time video feed of the at least a portion of the user shows only a modified portion of the user, such that (i) the modified portion of the user is visible to the user in the display and (ii) an unmodified portion of the user is visible to the user in the mirror.

Alternative Implementation 122. The system of Alternative Implementation 118, wherein the modified continuously displayed real-time video feed of the at least a portion of the user shows a modified portion of the user and an unmodified portion of the user, such that neither the modified portion of the user or the unmodified portion of the user are visible in the display.

Alternative Implementation 123. A system for displaying images or videos, the system comprising: a frame; a mirror coupled to the frame; an electronic display device coupled to the frame such that the electronic display device is positioned adjacent to a portion of the mirror; a camera coupled to the frame, the camera having a field of view; and one or more processors configured to execute instructions stored in one or more memory devices such that the system is caused to: monitor, using the camera, a user positioned within the field of view of the camera; and continuously display, on the electronic display device, a real-time video feed of a first portion of the user, such that a second portion of the user is visible to the user in the mirror; receive a prompt; and subsequent to the receiving of the prompt, modify the displayed first portion of the user in the continuously displayed real-time video feed.

Alternative Implementation 124. The system of Alternative Implementation 123, wherein the prompt includes identifying an object located within the field of view of the camera.

Alternative Implementation 125. A system, the system comprising: a frame; a mirror coupled to the frame; an electronic display device coupled to the frame such that the electronic display device is positioned adjacent to a portion of the mirror; a camera coupled to the frame, the camera having a field of view; and one or more processors configured to execute instructions stored in one or more memory devices such that the system is caused to: monitor, using the camera, a user positioned within the field of view of the camera; continuously display, on the electronic display device, a real-time video feed of at least a portion of the user; determine a mood of the user; and subsequent to the determination of the mood of the user, modify the continuously displayed real-time video feed based on at least the determined mood of the user.

Alternative Implementation 126. The system of Alternative Implementation 125, wherein the modified continuously displayed real-time video feed of the at least a portion of the user shows a modified version of the user based on the determined mood of the user.

Alternative Implementation 127. The system of Alternative Implementation 103, wherein the mirror has a rectangular shape with a long dimension of between about four inches and about twelve inches.

Alternative Implementation 128. The system of Alternative Implementation 103, wherein the mirror has a circular shape with a diameter of between about four inches and about twelve inches.

Alternative Implementation 129. The system of Alternative Implementation 103, wherein the mirror has a square shape that is (i) about three inches by about three inches, (ii) about four inches by about four inches, or (iii) about five inches by about five inches.

Alternative Implementation 130. The system of Alternative Implementation 103, wherein the mirror is sized to fit a reflection of (i) an upper body of the user, (ii) a head and a neck of the user, or (iii) only a head of the user.

Alternative Implementation 131. The system of Alternative Implementation 103, wherein the mirror has a size and shape that generally matches a size and a shape of the electronic display device.

Alternative Implementation 132. The system of Alternative Implementation 103, wherein the one or more processors are configured to connect with one or more external devices, and wherein the one or more memory devices are configured to receive data from each of the one or more external devices.

Alternative Implementation 133. The system of Alternative Implementation 132, wherein the one or more external devices includes at least one of a mobile phone, a laptop computer, a desktop computer, an online server, a fitness tracker, a scale, a cloud service, an Internet-connected water bottle, an Internet-connected thermostat, a smart watch, a smart speaker, or any combination thereof.

Alternative Implementation 134. The system of Alternative Implementation 132, wherein the one or more processors are configured cause the electronic display device to display information related to first data received from a first of the one or more external devices, and related to second data received from a second of the one or more external devices

Alternative Implementation 135. The system of Alternative Implementation 132, wherein the one or more processors are configured to (i) analyze first data received from a first of the one or more external devices and second data received from a second of the one or more external devices, and (ii) correlate a portion of the first data with a portion of the second data.

Alternative Implementation 136. The system of Alternative Implementation 135, wherein the one or more processors are configured cause the electronic display device to display information related to the portion of the first data and the portion of the second data.

Alternative Implementation 137. The system of Alternative Implementation 135, wherein the one or more processors are configured cause a recommendation to be communicated to the user, the recommendation being based at least in part on the first portion of the first data and the second portion of the second data.

It is contemplated that any elements of any one of the above alternative implementations can be combined with any elements of one or more of any of the other alternative implementations and such combinations are intended as being included in the scope of the present disclosure.