Abstract:
The present invention provides an automatic cooking method and system, wherein the cooking process of a chef is recorded. Then, a program about the cooking process is obtained with information about amounts and kinds of main ingredients and seasoning materials used by the chef, timing of adding main ingredients and seasoning materials and movement tracks of the cooking container and shovel. Thereafter, manipulators of a mechanical operating system of the present invention imitate chef&#39;s cooking process according to commend signal from the program to produce a dish. The present invention uses recording devices to record chef&#39;s cooking process and provide a program, then respective mechanical operating system accomplishes cooking tasks imitating the chef, which provides restaurants and households with dishes by the chef when using the program and mechanical operating system. The present invention not only made exceptional dishes widely available, it can also serve a large number of patrons at the same time

Description:
FIELD OF THE INVENTION  
         [0001]    This application is a continuation in part of the PCT application PCT/CN02/00665 filed on Sep. 18th, 2002. The present invention relates generally to an automatic cooking method and system, and more particularly to a method and system of recording and simulating a cooking process for a dish by a famous chef.  
         BACKGROUND OF THE INVENTION  
         [0002]    Traditionally, cooking is done by chefs one dish at a time. Exceptional dishes sometimes can only be produced by a limited number of famous chefs. Therefore, certain dishes cannot be provided unless a particular chef is invited to a restaurant. For example, Chinese cuisine has many types of cooking styles. For each style there are only limited numbers of chefs who are capable of producing certain exceptional dishes. In the past, the styles and crafts were passed on to the next generation individually, but this method has obvious limitations. It cannot serve a large number of patrons at the same time. The public is not well served by this method.  
           [0003]    The objective of the present invention is to provide an automatic cooking method and system that will widely spread the unique techniques of the famous chefs.  
         SUMMARY OF THE INVENTION  
         [0004]    In order to achieve the objective of the present invention, the present invention provides an automatic cooking method including the following steps:  
           [0005]    1). Measuring and recording kinds and amounts of main ingredients and seasoning materials prepared by a chef;  
           [0006]    2). Measuring and recording movement tracks of cooking containers and shovels;  
           [0007]    3). Measuring and recording amounts of main ingredients and seasoning materials added by the chef and timing when the main ingredients and seasoning materials are added;  
           [0008]    4). Measuring and recording strength of fire adjusted by the chef and timing when strength of fire is adjusted;  
           [0009]    5). Processing the data recorded by step 1 through step 4 by a computer and providing an operation program of the automatic cooking system that reflects the chef&#39;s cooking process;  
           [0010]    6). Installing the operation program into the automatic cooking system and central control device, enabling the central control device to operate according to the operation program and to complete the cooking process.  
           [0011]    Additionally, in order to achieve the objective of the present invention, the automatic cooking system provided by the present invention comprises recording system that records the chef&#39;s cooking process, and mechanical operation system that operates according to recorded cooking process, wherein said recording system comprises:  
           [0012]    1). Equipment for measuring and recording kinds and amounts of main ingredients and seasoning materials prepared by a chef;  
           [0013]    2). Equipment for measuring and recording amounts of main ingredients and seasoning materials added by the chef and timing that main ingredients and seasoning materials are added;  
           [0014]    3). Equipment for measuring and recording movement tracks of cooking containers and shovels;  
           [0015]    4). Equipment for recording and measuring strength of fire adjusted by the chef and timing when main ingredients and seasoning materials are added;  
           [0016]    5). Equipment for processing recorded data, and for providing operation program of the automatic cooking system that reflects the chef&#39;s cooking process.  
           [0017]    The mechanical operation system includes:  
           [0018]    1). A controller installed with the operation program that reflects the chef&#39;s cooking process;  
           [0019]    2). Manipulators that is adapted to be connected with the cooking containers and shovels, and complete the cooking tasks according to signals received from the operation program of the controller that reflect the chef&#39;s cooking process;  
           [0020]    3). Fire controlling device that is connected with the stove, and control the strength of fire according to signals received from the operation program of the controller that reflect strength of stove controlled by the chef;  
           [0021]    4). Main ingredient and seasoning material supply devices that is connected with the main ingredient and seasoning material supply equipment, and control the main ingredient and seasoning material supply equipment according to signals received from the operation program of the controller that reflect the kinds and amounts of the main ingredients and seasoning materials added by the chef and timing when the main ingredients and seasoning materials are added.  
           [0022]    The present invention records the cooking process of the chef with recording system and provides operation program according to the cooking process, then it operates the mechanical operation system to complete cooking tasks. Thus, delicious dishes by famous chefs will be available to any restaurant or household who purchase the mechanical operation system and operation program. This will not only widely spread the technique for making those exceptional dishes, it will also save manpower. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0023]    [0023]FIG. 1 illustrates the recording system for recording the cooking process of a chef;  
         [0024]    [0024]FIG. 2 illustrates the mechanical operating system of the automatic cooking system;  
         [0025]    [0025]FIG. 3 illustrates the base coordinate system, camera lens coordinate system and cooking container and shovel coordinate systems;  
         [0026]    [0026]FIG. 4 illustrates the positions of the specific points in camera lens coordinate system;  
         [0027]    [0027]FIG. 5 illustrates the positions of the specific points at the camera receiving membrane;  
         [0028]    [0028]FIG. 6 is the calculation of the position and posture of a cooking container based on the coordinates of the specified points at base coordinate system;  
         [0029]    [0029]FIG. 7 is the block diagram of calculation of the movement track of the manipulators;  
         [0030]    [0030]FIG. 8 illustrates the dynamic model of the manipulators;  
         [0031]    [0031]FIG. 9 illustrates the dynamic model of allover components of the mechanical operating system of the automatic cooking system;  
         [0032]    [0032]FIG. 10 is the electrical circuit drawing of the recording system that records the demo cooking process of the chef;  
         [0033]    [0033]FIG. 11 is the electrical circuit drawing of mechanical operating system of the automatic cooking system;  
         [0034]    [0034]FIG. 12 is the manipulator control diagram.  
     
    
       [0035]    Illustrated below is the detailed description of the embodiment of present invention related to the Figures.  
       DETAILED DESCRIPTION OF THE INVENTION  
       [0036]    [0036]FIG. 1 illustrates the cooking process of a chef. As illustrated in FIG. 1, multiple (three were shown) main ingredient containers  8  are used to contain the main ingredients of the dish. Two cameras  1  record the cooking process conducted by the chef, said cameras  1  receive images of the cooking process conducted by the chef and send the signals to a computer  3 . Said computer  3  processes the images. The computer  3  measures and calculates the coordinate system positions of each specified points P of cooking container (such as wok)  10  and shovel  11  during the movement in the cooking process (see FIG. 3), and obtain the movement track of cooking container  10  and shovel  11 . Multiple (four were shown) seasoning material containers  6  contain seasoning materials such as cooking oil, salt, sauces, and vinegar, etc. An electronic scale  7  is placed under each seasoning material container  6  respectively, the electronic scales  7  measure the amounts of seasoning materials added to the cooking container  10  and transform the data to electronic signals. In the mean time, said electronic scales  7  also record the time when the seasoning materials are added. These data are transformed into numerical signals through first interfaces (such as A/D board, I/O port)  9 A and sent to computer  3  (see FIG. 10). Therefore, the data for the amounts of seasoning materials added and the time when the seasoning materials are added will be stored in computer  3 . The rotation sensor set in stove  4  will measure the rotated angle of stove rotating switch  5 . Said rotation sensor will record the adjustment of the strength of stove by the chef, and simultaneously record the time when the adjustments are made. A temperature sensor (such as thermocouple thermograph) installed on the cooking container (such as wok or pot) can also measure the temperature of the cooking container to reflect the strength of the fire, the signal for the temperature will also be sent to computer  3 . A position sensor (such as infrared transmitter, or touch on switch)  12  can also record the time when the main ingredients are put into the cooking container  10 .  
         [0037]    [0037]FIG. 2 illustrates the mechanical operating system of automatic cooking system. As shown, the present invention uses one or more manipulators and affiliating devices to construct mechanical operating system. The automatic cooking system comprises multiple, such as three, seasoning material containers  17 , the output of which are controlled by a computer program and program controlled funnels  18  placed underneath that can add the seasoning materials for the dish according to operation commends of the program. Each manipulator  21  of the mechanical operating system has at least one mechanical hand  16 , and has at least six degrees of freedom (i.e. has at least six joints  20 ). Said mechanical hands  16  can grab and hold multiple, such as three, main ingredient containers  15  that contain main ingredients of the dish and pull out the main ingredients according to the operation commends of the program. The stove switch  19  can control the fire strength of the stove according to operation commends of the program. The central controller  14  comprises exterior storage interface  31  and displayer  32 . The exterior memory (such as diskette, CD, etc.) containing operation program is installed into central controller  14  (or the operation program is sent to central controller  14  through network). Once the main ingredients and seasoning materials for the dish are prepared according to the requirement of the operation program, said mechanical operating system will automatically start cooking once the operation program starts.  
         [0038]    [0038]FIG. 3 illustrates the setup of base coordinate system Σ b , camera lens coordinate system (Σ cl and  Σ cr ) and cookware coordinate system of the recording system. The base coordinate system Σ b  of the recording system is correspondence to the base coordinate system Σ b  of the mechanical operating system. Through this setup, the intended movement tracks of the cooking container  10  and shovel  11  can be obtained by the operation program. As shown in FIG. 3, two of the camera coordinate systems set at the centers of the lenses of the left and right cameras  1  respectively, and the positions of said two cameras and the positions of base coordinate system are directly related. Said camera coordinate systems are shown as Σ cl,  Σ cr  (also shown in FIG. 4). Two movement coordinate systems set at the tips of the handles of the cooking container  10  and shovel  11  are presented as Σ g,  Σ q . The coordinate system positions of said cooking container  10  and shovel  11  directly relate to the coordinate positions of three specified points P (dots of same color). Calculations based on the images recorded by the cameras provide the relevant positions of the specified points to the camera coordinate systems, and the relevant positions of Σ g,  Σ q  to base coordinate system Σ b  can also be calculated. The details of the calculation are illustrated below.  
         [0039]    As shown in FIG. 4, one specific point P is used as an example for the calculation, wherein camera  1  comprises image receiving membrane  33 , camera lens  34 . The axis center lines cr and cl of left and right cameras  1  are parallel with each other, and in line with said camera lens  34 . The centers of lenses O cl  and O cr  are origins of the camera coordinate system, and the line connect them becomes Y axis of the camera (the Y axis of the left camera is y cl,  the Y axis of right camera is y cr ). The axis center lines of two cameras are X axis, wherein the X axis of left and right cameras is x cl, x   cr  respectively. The plane formed by x cl  and y cl  of the left camera are the same with the plane formed by x cr  and y cr.  Therefore, the left and right cameras&#39; center lines are at the same plane. The z cl  and z cl  axis of the left and right cameras are perpendicular to this plane, which is called Z plane. The plane formed by y cl  and z cl  axis of the left camera must be the same as the plane formed by y cr  and z cr  axis of the right camera, that is the left and right camera are in line with each other. The plane formed by x cl  and z cl  axis of the left camera is parallel to the plane formed by x cr  and z cr  axis of the right camera, and distance between them is w, which is also the distance between the origins of the coordinate system O cl  and O cr.  The distances of point P in FIG. 4 to each plane at each moment are its positions at the left and right camera coordinate systems, shown as ( cl x p ,  cl y p,    cl z p ,  cr x p ,  cr y p ,  cr z p ).The projection point from point P to plane Z is P z . Connecting PO cl,  PO cr. , and P z O cl  and P z O cr,  the angle between PO cl  and P z O cl  is called  cl θ Py , the angle between PO cr  and P z O cr  is called  cr θ Py , the angle between P z O cl  and x cl  is called  cl θ Pz , the angle between P z O cr  and x cr  is called  cr θ Pz , and the relationship between them can be calculated by the equation below, wherein  cl ,  cr  represent the coordinate systems of the left and right cameras: 
           cr x p tan  cr θ Pz = cr y p   
           cl x p tan  cl θ Pz = cl y p   
         Because  cl   x   p = cr   x   p , and  cl   y   p + cr   y   p   =w,   
           cr   x   p  tan  cr θ Pz + cl   x   p  tan  cl θ Pz   =w   
           cr   x   p = cl   x   p   =w (cot  cr θ Pz +cot  cl θ Pz ) 
           cr   y   p = cr   x   p  tan  cr θ Pz,   cl   y   p = cl   x   p  tan  cl θ Pz   
         [0040]    [0040]         z   p               cr     =       z   p               cl     =       x   p               cl          1     sin        θ   Pz               cl            tan        θ   Py               cl                                 
         [0041]    [0041]FIG. 5 illustrates how to use the positions of  sr P ( sr x p ,  sr y p ) (the measuring unit of which needs to be transformed from pixel into mm, the transformation ratios are different for different cameras) in the image received by image receiving membrane  33  of the right camera to obtain rotation angles ( cr θ Py ,  cr θ Pz ).  sl ,  sr  represent the image receiving membrane plane coordinate system of the left and right cameras, and l is the distance between image receiving membrane and the center of the lens.  
         [0042]    The equation for calculating the rotation angle is shown below:  
                 θ   Pz               cr     =       -     tan     -   1                x   p               sr     l                                  θ   Py               cr     =       -     tan     -   1                  y   p               sr        cos        θ   Pz               cr       l                                   
 
         [0043]    [0043] cl θ Pz  and  cl θ Py  can also be calculated from  sl x p  and  sl y p  of left camera using the same method.  
         [0044]    [0044]FIG. 6 illustrates how to use the positions of three specific points ( b             g1 ,  b             g2 ,  b             g3 ) on the cooking container to calculate the related position ( b             g ) and posture ( b             g ) of the cooking container coordinate system Σ g  to the base coordinate system Σ b . The position and posture Matrix of the cooking container ( b             g ) can be calculated as shown below: 
           b             g1 =[ b x g1 , b y g1 , b z g1 ] 
           b             g2 =[ b x g2 , b y g2 , b z g2 ] 
           b             g3 =[ b x g3 , b y g3 , b z g3 ] 
         [0045]    [0045]         T   g               b     =     [           R   g               b           p   g               b               0   T         1         ]                           
  b             g =           g(   b P g1 , b P g2,   b P g3 ) 
         [0046]    Wherein  b  is the base coordinate system.  b x g1  represents the distance of point P 1  on the cooking container  10  at base coordinate system Σ b  to plane y b -z b . Because camera coordinate system and base coordinate system has fixed relationship,  b             gn  (n=1, 2, 3) may be obtained from ( cr x pg,    cr y pg ,  cr z pg ) and ( cl x pg,    cl y pg ,  cl z pg ) through coordinate transformation.            g  in the equation is linear algebra function. The function of            g  is obtained according to the setup of the cooking container coordinate system, and a person with ordinary skill in the art can obtain it according to a well known calculation method, which will be omitted here. The result of the above calculation will be shown at the block diagram of FIG. 7, wherein the calculation process in FIG. 7 is illustrated as follow: The coordinate positions of the three specified points ( sr             1 ,  sr             2 ,  sr             3  and  sl             1 ,  sl             2 ,  sl             3 ) on the image receiving membrane plane coordinate systems (x sr −y sr  and x sl −y sl ) are obtained using computer image processing, then the coordinate position of each specified point ( cr             1 ,  cr             2 ,  cr             3  and  cl             1 ,  cl             2 ,  cl             3 ) in each camera lens coordinate system (x cr −y cr −z cr  or x cl −y cl −z cl ) is calculated through coordinate transformation. Further, the coordinate position of each specified point related to base coordinate systems is calculated through coordinate transformation, and the relative position and posture of the cooking device coordinate system Σ g  to the base coordinate system Σ b,  refereed to as  b T g (k), are obtained, where k is the sample cycle number.  
         [0047]    As shown in FIG. 7, “other action signals”, such as action signals indicating the action of taking the ingredients or adjusting the strength of the fire, are inserted. Because these actions are decided according to starting points and ending points, and the calculations are relatively simply, and a person of ordinary skill in the art will be able to calculate them easily, the calculation will be omitted here. Each manipulator has its own specific kinematics function, and            r  is the kinematics function of the right manipulator.            r (k) is the rotation angle vector of each joint of the right manipulator (also called object track). The calculation process of the rotation angle vector of the each joint of the left manipulator is the same with the right manipulator, which will be omitted here. The calculation for shovel is the same as for the cooking container, which will also be omitted here.  
         [0048]    Because the image processing process for the cooking process recording system is only used to identify some specified marks, ordinary image identification method such as Hough Transform can be used to calculate the position of each mark, which will not be discussed in detail here. Besides, the method of recording the cooking process illustrated above can also be replaced by Direct Teaching method often used in ordinary manipulator (the chef can use his/her arms to direct manipulators in order to directly teach the manipulator the movement for the cooking process), or replaced by Magnetic Field sensor (magnetic sensors can be used to record the positions of the cooking container and shovel in order to record the cooking process). However, the method illustrated here will be able to guarantee accuracy. In practice, these methods can compliment each other. The calculation and manipulator operating program are performed by computer automatically. The image processing and camera recording process may be conducted simultaneously, or the image processing task can be conducted after the images have been recorded.  
         [0049]    [0049]FIG. 8 is the structure and kinematics model of the manipulators of the automatic cooking system. As shown in FIG. 8, the structures of these manipulators are similar to the structure of ordinary manipulators for industrial use. θ 0l ,θ 2l  . . . θ 5l  and θ 0r , θ 1r  . . . θ 5r  represent the rotation angles of each of the six elbows of two manipulators.  
         [0050]    [0050]FIG. 9 is the kinematics model of all the components of the mechanical operation system of the automatic cooking system. The setups of the coordinate system are as follow: The setups of the coordinate systems of the cooking container and shovel (Σ g , Σ q ) and the base coordinate system Σ b  are the same as those of the coordinate systems of the cameras recording chef&#39;s operation. The directions and positions of the coordinate systems of main ingredient containers  15  (Σ v1 , Σ v2  . . . Σ vn ) are set that when any of the mechanical hands  16  holds any of the main ingredient containers  15 , the coordinate system of this mechanical hand is superimposed upon the coordinate system of the respective main ingredient container  15 . The directions and positions of the coordinate systems of the seasoning material containers  17  (Σ f1 , Σ f2  . . . Σ fu ) are set that when the cooking container  10  receives the seasoning material from any of the seasoning material containers  17 , the coordinate system of the cooking container  10  is superimposed upon the coordinate system of the respective seasoning material container  17 . The stove fire switch will be controlled by a computer program controlled motor.  
         [0051]    [0051]FIG. 10 is the electrical circuit drawing of the recording system for recording a Chef&#39;s cooking process. As shown in FIGS. 1 and 10, cameras  1  are connected with image processing circuit boards  2 . Said cameras  1  will send the images of the cooking process of the Chef through image processing circuit boards  2  to computer  3 . The stove rotating switch  5  of the stove having a rotation sensor is connected with the first interface  9 A (such as A/D circuit board). Said stove rotating switch  5  records the signals of the rotating angle (the strength of the fire) chosen by the Chef, and sends the signals through the first interface  9 A to computer  3 . The seasoning material containers  6  comprise electronic scales  7 , and the changed amounts of the seasoning materials used by the Chef will be recorded by the electronic scales  7 . These data will be sent through the first interface  9 A to computer  3 . The main ingredient containers  8  comprise ingredient sensors  12  (such as diode infrared transmitters) that are connected with the second interface  9 B. Said ingredient sensors  12  will send signals representing the time when the main ingredients are put into the cooking container through the second interface  9 B to computer  3 . A simple program modular can be made according to the above data. According to the result of the above calculation, computer  3  having a ready-made program in addition to program modular will provide operation program imitating chef&#39;s movement when cooking each dish. The program can be adjusted if necessary according to the requirement of each occasion.  
         [0052]    [0052]FIG. 11 is the electrical diagram of the mechanical operating system of the automatic cooking system. After the operation program of the automatic cooking system is inserted into computer  35  of the central controller, the system will be started. Computer  35  will send operation program&#39;s control signals for controlling the amounts of seasoning materials through pulse generating board  22  to stepping motor drives  23 , and then the stepping motor drives  23  motivate the stepping motors of the program controlled funnel  18  of the seasoning material containers  17 , and add the seasoning materials to the cooking container. Computer  35  will send the control signals of the operation program that control the strength of stove fire through pulse generating board  22  to respective stepping motor drive  23 . The stepping motor drive  23  will control stepping motor of the program controlled stove rotating switch to control the strength of the fire. The operation program of the computer  35  will control mechanical hands  16  to grab main ingredient containers and pour out main ingredients. The control signals will be sent through pulse generating board  24  to stepping motor drives  25 . The stepping motor drives  25  will control mechanical hands  16  to grab and hold main ingredient containers, cooking container, and shovel. The movements of each joint of the manipulators when accomplishing the cooking tasks are controlled by the computer. The computer sends control signals through a third interface  26  (such as A/D circuit board) to drivers  27 . The drivers  27  will control the movement of the manipulators through the motors of each joint  20  to accomplish cooking tasks. The angle rotation signals of the manipulators&#39; motor angle rotation transmitters will be amplified through amplifier  29  and send to counter board  28 . The feedback signal of the movement of the manipulators will be sent to computer  35 , in order to calibrate the movements of the manipulators. The manipulators of this system will be able to imitate the cooking process according the operation program.  
         [0053]    [0053]FIG. 12 is the manipulator control diagram, wherein θ r (k) is the target movement track of the right manipulator calculated from operation recording system (the output of FIG. 7), same is for the left manipulator. Said block diagram is the PID control diagram for ordinary controlled manipulators. Once the target track is known, each manipulator can be controlled as shown in FIG. 12. θ ro (k) is the actual rotated angle of each joint of the manipulator, and θ′ ro (k) is the angle velocity of each joint of the manipulator.  
         [0054]    As illustrated above, the essence of the present invention is to record and process the images of the movement of the chef when cooking (or real time process). Through calculations of the coordinate positions of three specified points on the cooking container and shovel used by the chef, the movement tracks of the cooking container and shovel when used by the chef to cook certain cuisine can be obtained (although the drawings and detailed description only referred to one cooking container and one shovel, the same method can be used for more than one cooking container and shovel). Then, the cooking program can be produced according to the amounts of main ingredients and seasoning materials added by the chef and the time the main ingredients and seasoning materials are added when cooking, the strength of the stove fire and the movement track of the cooking container and shovel. Furthermore, the manipulators of the mechanical operating system of the present invention will imitate the chef&#39;s cooking movements according the signals of the above program and produce the cuisines similar to what are produced by the chef. In addition, the method and system of the present invention can also be used partially. For example, recording system or mechanical operation system may be used as an independent system. Or, the mechanical operation system may be used as a supplemental device supporting chef&#39;s operation.