Patent Publication Number: US-2023147029-A1

Title: Systems and methods for automated makeup application

Description:
TECHNICAL FIELD 
     The present disclosure relates to systems and methods for automated makeup application, and more particularly, to systems and methods for applying makeup to a face without continuous human intervention. 
     BACKGROUND 
     Applying makeup can be very time consuming, often taking several hours a week. It may also be unhygienic since makeup and brushes are often not cleaned properly after each use, which can lead to acne and infections. Makeup can also be very expensive as a user may purchase multiple products from many different makeup brands, colors and styles to achieve a desired look. Whether a user is applying makeup to oneself or is getting a makeover in a salon, poor color matching often results from the lack of available foundation shades. Despite the identification of over 100 skin colors, most makeup companies only offer less than 40 foundation shades, and even fewer options for women of color. Therefore, imperfect or off-color foundation is a frequent occurrence, and unintentionally discriminates against women of color. 
     Traditional makeup application methods, products, and implements are relied upon by a cosmetics industry worth many billions of dollars in sales every year. However, there is still a demand for new products and methods to help consumers save time and money and improve their hygiene when applying makeup. 
     SUMMARY 
     The present disclosure relates to systems and methods for automated makeup application that allows a user to select a desired makeup style and apply the selected makeup to the user&#39;s face. In accordance with the present disclosure, a method for automated makeup application is provided which substantially eliminates or reduces disadvantages and problems associated with previous systems and methods. 
     In accordance with one aspect of the present disclosure, a method is provided for recording a face map and facial colors of the user; choosing a desired look; calculating formula quantity needed for each makeup category of the selected look; collecting and releasing the formula into a reservoir; mixing the formula; spraying the user&#39;s face with formula through an airbrush nozzle; and cleaning the reservoirs and nozzles. 
     An embodiment of the present disclosure includes a method and system for automated makeup application that allows the user to choose a look from a plurality of preconfigured looks based on personal preference. By automating the makeup application process to achieve an appropriate look, and thereby reducing the need to apply makeup by hand, the system saves time and reduces delays. The system also reduces the time wasted on gathering and preparing various cosmetics by automating the process of calculating formula amounts, collecting formula, and releasing it for application. The system is particularly useful for aiding those that find decisions regarding the selection of an appropriate look, makeup style, and colors difficult or burdensome. Furthermore, the system is especially useful in aiding persons with physical disabilities. 
     Another important aspect of the present disclosure is the emphasis on personal hygiene of the user. By eliminating the practice of using cosmetics, brushes, and other related implements without properly cleaning them between each use, as is the case in traditional makeup application, the present disclosure protects users from the threat of contracting diseases, germs, and infections. 
     A further important aspect of the present disclosure is the accuracy in color matching and foundation application. Matching different shades of color from thousands of different brands and colors is a challenging task and requires considerable training and experience to efficiently combine these colors to achieve the desired look manually. By automating this task, the present disclosure accurately applies makeup along the lines of the color template chosen by the user. The system is particularly useful for aiding those that find it difficult to choose the foundation that is appropriate for their skin tone, since the makeup industry does not offer enough foundation shades to match a user&#39;s exact skin tone. Further automating this task, the system records a face map which includes the user&#39;s facial colors and determines the foundation that is appropriate for the user&#39;s skin tone. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       For a more complete understanding of this disclosure and its features, reference is now made to the following description, taken in conjunction with the accompanying drawings, in which: 
         FIG.  1    illustrates a perspective view of an enclosed automated makeup application system and its removable components in accordance with one implementation of the present disclosure. 
         FIG.  2    illustrates a perspective view of various components disposed within the automated makeup application system in accordance with one implementation of the present disclosure. 
         FIG.  3    illustrates an enlarged perspective view of various components of a pump/optic sensor module of the automated makeup application system of  FIG.  2   , in accordance with one implementation of the present disclosure. 
         FIG.  4    illustrates an enlarged perspective view of various components of a robotic arm of the automated makeup application system of  FIG.  2   , in accordance with one implementation of the present disclosure. 
         FIG.  5    illustrates a perspective view of the interior motors and hose of the robotic arm of  FIG.  4   , in accordance with one implementation of the present disclosure. 
         FIG.  6    illustrates a diagram of an assembly for controlling the automated makeup application system of  FIG.  2    with user data, in accordance with one implementation of the present disclosure. 
         FIG.  7 A  and  FIG.  7 B  illustrate a flowchart of a method for automated makeup application, in accordance with one implementation of the present disclosure. 
     
    
    
     DETAILED DESCRIPTION 
     Traditional makeup application methods tend to be messy, time-consuming, costly, and unhygienic. The present disclosure is directed to quick, sanitary, and cost-effective systems and methods for makeup application. 
     The systems and methods of the present disclosure substantially eliminate human intervention in makeup selection, appropriation, application, and cleanup, while performing these tasks accurately, cost-effectively, and in an acceptable hygienic manner. 
     The present disclosure represents unique systems and methods for applying makeup without substantial or continuous human intervention. One implementation of the present disclosure employs a computer and computer code or software and is capable of operation over the internet when hosted by a cloud-based server. 
     The present disclosure relates to systems and methods for automated makeup application that allow a user to select a desired makeup style and apply the makeup associated with the selected makeup style to the user&#39;s face. Accordingly, the systems and methods of the present disclosure will save users time spent in carrying out the several sub-processes associated with makeup application; (a) gathering various cosmetics, (b) choosing a look to apply to the user&#39;s face, (c) choosing the colors required to acquire the look, (d) applying each category of cosmetic, for example, eyeshadow, eyeliner, or lip stick, independently to achieve the desired look based on different formulas of makeup, and (e) cleaning up and putting away the various cosmetics. The present disclosure combines all of these sub-processes into one action as far as the user is concerned, i.e. choosing a look. 
     The present disclosure serves to make the process of applying makeup more hygienic. A conventional makeup application process employs the use of brushes which come into physical contact with the user&#39;s skin. The same brush may be used over a long period of time and it can be very unhygienic because most people don&#39;t appropriately clean their makeup and brush after each use. Bacteria can build up over time and cause acne or an even more serious condition, for example, spreading pink eye or other infection due to re-use of makeup that came in contact with a bacteria. The present disclosure employs air brush technology, which eliminates the possibility of contamination caused by unsanitary makeup, cosmetic brushes and other implements. 
     Furthermore, the present disclosure will substantially eliminate the likelihood of poor color matching and improper foundation application. Perfect color matching and exact application technique are difficult to achieve by a user possessing beginner or intermediate skills in makeup application. Experienced make-up artists charge hundreds of dollars to provide an accurate and desired look. The present disclosure provides users a cost-effective and automated solution by mixing distinct colors to reach a desired shade and applying the right foundation to the appropriate skin tone. 
     Referring to  FIGS.  1  and  2   , in one implementation of the present disclosure, an automated makeup application system  100  comprises a casing  10  with a retractable lid  12 , which provides access to the inner contents of the automated makeup application system  100 , as shown in  FIG.  2   . A formula sleeve  14  and a cleaning pad  30  are removable through appropriate slots on the side of casing  10 . Formula sleeve  14  is adjacent to a side of casing  10  and comprises a plurality of pods  16  which are used to store formula. A pair of timing pulleys  22  are coupled to the floor of casing  10  and stand vertically on both ends of the formula sleeve  14 . A pump/optical sensor module  17  stands vertically next to the formula sleeve  14 . The pump/optical sensor module  17  further comprises a pump  18 , a cylindrical tube  21 , a ledge  19 , and an optical sensor  20 , as shown in detail in  FIG.  3   . The pump/optical sensor module  17  is embedded in a formula track  26  which runs along the length of the formula sleeve  14 . The timing pulleys  22  push and pull the pump/optical sensor module  17  with the aid of a belt clamp  24  along the formula track  26 . A compressor  28  sits adjacent to the formula sleeve  14  and is used to mix formula. A servo-motor  32  lies adjacent to another side of the compressor  28  and is used to control movement of other components. A microcontroller  34  sits opposite the compressor  28  on the right side of casing  10  and controls various devices and processes. An airbrush compressor hose  36  runs along the floor of casing  10  and connects the compressor  28  to a robotic arm  40 , which is illustrated in detail in  FIGS.  4  and  5   . 
     Referring now to  FIG.  3   , in one implementation of the present disclosure, the pump/optical sensor module  17  comprises a pump  18 , a cylindrical tube  21 , a ledge  19 , and an optical sensor  20 . The pump  18  sits at the top of the pump/optical sensor module  17 . It hovers over pods  16  and collects required amounts of formula when instructed. A servo-motor  38  is coupled to the pump  18  and controls its movement. The cylindrical tube  21  is coupled to pump  18  at its top end and ledge  19  at its bottom end and helps the pump/optical sensor module  17  stand vertically adjacent to formula sleeve  14 . Extending away from cylindrical tube  21 , ledge  19  runs below formula sleeve  14 , parallel to the floor of casing  10  and pump  18 . The ledge  19  is a resting spot for the optical sensor  20  which sits on top of ledge  19  and is right underneath formula sleeve  14 . The optical sensor  20  is located underneath formula sleeve  14  and scans pods  16  for available formula. 
     Referring now to  FIG.  4   , a perspective view of robotic arm  40 , and  FIG.  5   , a distinct perspective view depicting both the internal and external components of robotic arm  40 . In one implementation of the present disclosure, robotic arm  40  is coupled to the floor of casing  10  with a bracket  44  which is coupled to the top of a base  42 , as shown in  FIG.  4   . The robotic arm  40  houses several other components and helps with several processes, including face mapping and makeup application. In one implementation of the present disclosure, robotic arm  40  comprises of two sub-components; arm  46  and arm  48 . Arm  46  which is topped off with a domed structure  47 , is coupled to bracket  44  and can stand vertically and parallel to the sides of casing  10  when instructed. Another arm  48  is coupled to the domed structure  47  of arm  46  and runs horizontally and parallel to the floor of casing  10 . Airbrush compressor hose  36  runs along the length of arms  46  and  48 . One end of arm  48  is coupled to a head  50  which houses several other components. A plurality of reservoirs  52  are situated towards the top of head  50 . A plurality of nozzles  54 , a camera  56 , and a photoelectric sensor  58  are located inside the head  50  so as to be visible on the surface of head  50  that faces away from the robotic arm; and that face of head  50  is bordered by a plurality of lights  60 . The reservoirs  52  store makeup formula delivered by pump  18 ; moreover, the formula is mixed inside the reservoirs  52  through back-bubbling. The nozzles  54  are used to spray formula to the user&#39;s face. The camera  56  is used to record the images of the face, which then sends the images to the phone in real time. The photoelectric sensor  58  is used to determine the distance to, absence, or presence of an object with the aid of a light transmitter. A servo-motor  62  is housed inside base  42 , as shown in  FIG.  5   . That motor can rotate arm  46  up to 360° when required. Several other servo-motors are also illustrated; servo-motor  64  sits inside the domed structure  47  connecting arm  46  and arm  48  and can rotate arm  48  in an upward or downward direction when required; and, servo-motor  66  is located at the far end of arm  48  and sits on top of head  50 . Airbrush compressor hose  36  is shown to run along the length of arms  46  and  48 . The head  50  further encloses a plurality of reservoirs  52 , a plurality of nozzles  54 , a plurality of needles  68 , and a plurality of plug triggers  70 . The plug triggers  70  are coupled to the needles  68  and work together to control the rate which makeup formula is sprayed. The plug triggers  70  move the needles  68  to allow makeup formula to be sprayed through the nozzles  54  when required and to completely block the nozzle airway and cease spraying. 
     Referring to  FIG.  6   , in operation, the user&#39;s first interaction with the automated makeup application system  100  is through a computer application, which can be accessed via mobile phone  92 , personal digital equipment, notebook, laptop, tablet computer, desktop computer, and/or an equivalent. A first-time user registers in the computer application  80  and a returning user logs into the computer application  80 . All user data, which may comprise logins, face map, skin tone, color and style preferences, are stored in a virtual server database (database)  90  which is, in one implementation, hosted in the cloud. Apart from user data, preconfigured looks and data for color combinations are also stored on the database  90 . 
     Referring to  FIG.  7 A  and  FIG.  7 B , one implementation of a method  700  for automatically applying makeup is schematically depicted. The method  700  begins on  FIG.  7 A  with step  702  during which a first-time user registers in the computer application, or a returning user logs into the computer application. In step  704 , all user data is stored in the virtual database. In step  706 , the user initiates the face mapping process in the computer application. Subsequently, the microcontroller  34  signals the robotic arm  40  to initiate operation to record the user&#39;s face map, skin tone, and lip color. The camera  56  mounted on the robotic arm  40 , begins mapping the user&#39;s face as the robotic arm  40  travels 180° around the user&#39;s face, moving to get a full-frontal view, then moving to one side for a side view, and to the other side of the face for another side view. In step  708 , the camera  56  records a face map along with the user&#39;s skin tone and lip color. In step  710 , the camera  56  sends that data to the database. 
     In step  712 , the computer application pulls preconfigured looks from the database and displays them to the user. In step  714 , the user chooses a look. In step  716 , a virtual face recognition software residing on the database adjusts the selected look&#39;s facial template to the user&#39;s facial map and skin tone. In step  718 , the computer application then calculates the formula quantity required for a plurality of makeup categories, including but not limited to; foundation, contouring, blush, highlighter, eyeshadow, eyeshadow highlighter, eyeliner, and lip color. The computer application also calculates how much the pump  18  needs to pump to obtain the quantity of each item needed. In step  720 , the computer application determines which cartridge numbers the pump  18  needs to move, the distance from its initial position to the appropriate pod  16 , and the distance from the pod  16  back to the appropriate reservoir  52  to release the formula. 
     In step  722 , with the calculations completed, the microcontroller  34  signals the optical sensor  20  to verify formula availability. Running along track  26  with the aid of timing pulleys  22  and belt clamp  24 , optical sensor  20  scans the appropriate pods  16  to check formula availability. In step  724 , when optical sensor  20  determines there isn&#39;t enough formula, the system triggers a low ink alert message, which is displayed by the application on the user&#39;s screen. Also, in step  724 , when optical sensor  20  determines there is enough formula, the system will clear makeup production. In step  726 , the pump  18  extracts formula from the appropriate pods  16 . In more detail, microcontroller  34  then signals timing pulleys  22  and belt clamp  24  to move pump  18  to the appropriate pod  16 . Pump  18  lowers its syringe into the pod  16  and triggers its pump to extract the precise quantity of formula from the pod  16 . In step  728 , the pump  18  releases the formula into appropriate reservoirs  52 . In more detail, pump  18  rotates away from pod  16  after extracting the formula and moves over to the head  50  of the robotic arm  40  and releases the formula into the appropriate reservoir  52 . The illustration of method  700  continues on  FIG.  7 B . In step  730 , pump  18  then rotates back over the appropriate pod  16 , lowers syringe, extracts formula, rotates back over the appropriate reservoir  52 , and releases the formula. The process is repeated until all required formulas is collected into corresponding reservoirs  52 . After all required formulas are deposited into corresponding reservoir  52 , microcontroller  34  signals airbrush compressor  28  to initiate operation. 
     In step  732 , on the head  50  of the robotic arm  40 , plug triggers  70  plug the airbrush system to back bubble the formula in order to mix each formula together to obtain the desired color. In step  734 , microcontroller  34  signals robotic arm  40  to activate by raising to its standing, neutral position with the aid of servo-motor  62 . In step  736 , the camera  56  begins live tracking the user&#39;s face. In more detail, microcontroller  34  signals camera  56  to begin live tracking of the user&#39;s face and signals the robotic arm  40  to move to its initial position in front of the user&#39;s face. In step  738 , the airbrush begins spraying the user&#39;s face. Specifically, microcontroller  34  signals plug triggers  70  to release the first airbrush nozzle  54  and begin spraying the user&#39;s face with formula. The system continues to track the location of robotic arm  40  in relation to the user&#39;s face, keeping the correct distance away from the face as robotic arm  40  moves, mimicking a pre-programmed human hand fashion. 
     In step  740 , microcontroller  34  signals robotic arm  40  to rotate back down to its starting position next to the floor of casing  10 . Reservoirs  52  and nozzles  54  are cleaned and disposed onto the cleaning pad  30  with cleaning solution. In step  742 , the process of formula extraction, disposal, back bubbling, face tracking, and application accomplished in steps  726  through  740  is repeated for each makeup category. 
     In step  744 , once the full face of makeup has been applied and all actions have been completed, the microcontroller  34  signals the robotic arm  40  to rotate back down to its starting position next to the floor of casing  10  and the lid on casing  10  is signaled to close. In step  746 , microcontroller  34  signals pump  18  to pick up cleaning solution from its corresponding pod  16 . In more detail, running along track  26  with the aid of timing pulleys  22  and belt clamp  24 , pump  18  lowers its syringe into pod  16 , and triggers its pump to extract the precise quantity of cleaning solution from pod  16 . In step  748 , pump  18  rotates away from pod  16  after extracting the cleaning solution and moves over to the head  50  of robotic arm  40  and releases the cleaning solution into the appropriate reservoir  52 . In step  750 , the process of back bubbling occurs again, and then microcontroller  34  signals the plug triggers  70  to release and spray the solution through the airbrush nozzles  54 , on to the cleaning pad  30 . In step  752 , after a pre-determined amount of uses, microcontroller  34  notifies the user that the cleaning pad  30  should be removed and replaced. 
     In step  754 , once all processes are completed, microcontroller  34  signals optical sensor  20  to verify formula availability. In more detail, running along track  26  with the aid of timing pulleys  22  and belt clamp  24 , optical sensor  20  scans the appropriate pods  16  to check formula availability. Where Optical sensor  20  determines there isn&#39;t enough formula, the system triggers a low ink message which is displayed by the computer application on the user&#39;s screen and prompts the user to replace the appropriate formula cartridge. Finally, in step  756 , microcontroller  34  triggers a makeup completion message which is displayed by the computer application on the user&#39;s screen.