Abstract:
A system, handheld devices, and methods for object tracking and control. The system tracks and controls items, people, and their interactions. The handheld mobile wireless device provides item specific information, instruction, and supervision. A plurality of handheld devices is configured to be used by different people, and communicate with a central database. A handheld device comprises a camera for scanning an image of an object, a display for displaying information, and a wireless network connection to the database for sending and receiving data. The system recognizes the object and identifies a person. The database includes information regarding people and their roles, permissions, and authorized or assigned tasks; objects and tools; object specific instructions; and the handheld devices. The instructions include documents, checklists, pictures, audio, and video. An observer, or the system, can monitor performance of the tasks, and provide corrective feedback.

Description:
BACKGROUND 
       [0001]    1. Field of Invention 
         [0002]    The present invention relates to object recognition and control, in particular the invention relates to object recognition and control including mobile wireless devices. 
         [0003]    2. Description of Prior Art 
         [0004]    Various attempts have been made to label objects with labels that can be read automatically by a computing machine. Barcodes have proven to be an effective scheme for achieving these purposes. Originally, linear or one-dimensional barcodes such as the Universal Product Code (UPC) were used. The UPC was the first commercially successful model and enjoyed widespread universal use. 
         [0005]    Later a two-dimensional or matrix code was developed. These are similar to one-dimensional codes, but represent more data per unit area and traditionally utilize square pixels. While the UPC represents a serial number, matrix codes can represent text, URLs, or other data. The Quick Response Code (QR Code or QRC) is one of the most popular types of two-dimensional codes and is often used with smart phones. 
         [0006]    More recently, a third-generation barcode called the High Capacity Color Barcode (HCCB) was developed. This two-dimensional barcode uses multi-colored triangles for an increase in data density, but still represents the same types of data. 
         [0007]    Bar code readers are typically connected to computer systems and databases to keep track of sales or inventory. 
         [0008]    These conventional methods have many drawbacks and limitations including the inability to identify individuals in various disciplines and roles and correlate relationships, tasks, permissions, etc. with other objects and/or individuals. In addition, these methods lack the ability to provide and present training, instructional and other information in various forms for maintenance and other purposes. 
         [0009]    There is a need for mobile wireless handheld devices to be able to recognize various objects and people and to provide for training, instruction, tracking, monitoring, and observation, which is authorized, and object specific. Further there is a need to be able to confirm that maintenance or use procedures are performed properly while they are being performed. 
       SUMMARY OF THE INVENTION 
       [0010]    It is an objective of the present invention to provide a means and method of identifying objects that include and go beyond those currently identifiable by conventional labels and go beyond existing tracking systems. Although traditional coded labels have proven to be an effective scheme for achieving item identification, they by themselves fall short of being able to achieve a more universal means and method of object recognition, training, instruction, tracking, monitoring, observation, and control. Handheld mobile wireless devices are used with objects and individuals to provide up-to-date pertinent information using methods such as on screen instructions, checklists, videos, pictures, diagrams, etc. or any combination thereof. Items and people are identified by scanning a label attached to the item or person, or by use of object recognition, facial recognition, or other biological recognition, such as iris scan, fingerprint scan, or other recognition known in the art. The system tracks and controls items, people, and their interactions. The handheld mobile wireless device provides item specific information, instruction, and supervision. 
         [0011]    A system aspect of the invention comprises a plurality of handheld devices used by different people, and a database in communication with the handheld devices. 
         [0012]    A handheld device comprises a camera for scanning an image of an object, a display for displaying information, and a wireless network connection to the database for sending and receiving data. 
         [0013]    The system recognizes the object by interpreting the image. In some embodiments, a coded label, e.g. a UPC or QR code, is interpreted. The system also identifies a person. The person may be identified by being associated with the handheld device: by scanning a scan label, such as an ID badge, ID card, or tattoo; or by scanning a face, a fingerprint, an iris, or other biometric. 
         [0014]    The database includes information regarding people and their roles, permissions, and authorized or assigned tasks; objects and tools; object specific instructions; and the handheld devices. The database also records information regarding the interaction between the objects and the people. 
         [0015]    The roles include maintenance, use, and observation, and may vary based on the object. 
         [0016]    The object specific instructions include documents, checklists, pictures, audio, and video. 
         [0017]    When the handheld device scans an object, the object is recognized, the person and their role are identified, and if the person is authorized the handheld device provides object specific instructions. 
         [0018]    In some situations, an observer is notified when use or maintenance is started by another person who is a user or a maintainer respectively. 
         [0019]    An observer, or in some embodiments, the automated system, can monitor the proper performance of the tasks, and provide corrective feedback during the performance. 
         [0020]    Method aspects of the invention include capturing an image with the camera of the handheld device, interpreting the image, determining an object, determining a person, and if the person is authorized determining a role of the person. 
         [0021]    A method of performing maintenance includes identifying a current part and any associated tools, providing object specific instructions for the current maintenance task, confirm completion of the task, and recording completion of the task, and repeating for each maintenance task. 
         [0022]    A method of performing use includes identifying tools, if any, providing object specific instructions for the current use task, confirm completion of the use, and recording information regarding the use, and repeating for each use task. 
         [0023]    A method of performing observation includes providing object specific instructions if required, and recording information regarding the start and finish of the observation, and repeating for each observation task. 
       OBJECTS AND ADVANTAGES 
       [0024]    Accordingly, the present invention includes the following advantages:
   1. To provide a means and method of object recognition and control.   2. To provide a means and method of recognition and control of people.   3. To provide a means and method of object recognition and control that has a common repository.   4. To provide a means and method of object recognition and control that is role based.   5. To provide a means and method of object recognition and control that is discipline based.   6. To provide a means and method of object recognition and control that is permission based.   7. To provide a means and method of object recognition and control that is used for verification and ratification.   8. To provide a means and method of object recognition and control that is used for assembly instructions.   9. To provide a means and method of object recognition and control that is used for maintenance notification, instructions, and tracking.   10. To provide a means and method of object recognition and control that is used to log maintenance and events.   11. To provide a means and method of object recognition and control that is used to track the events in life of an object.   12. To provide a means and method of object recognition and control that is used for reporting.   13. To provide a means and method of object recognition and control that is used for presenting information in a variety of mediums including documents, pictures, diagrams, video, audio, etc.   14. To provide a means and method of object recognition and control that is used for problem solving via knowledge bases that are associated with objects and individuals.   15. To provide a means and method of object recognition and control that is inexpensive and readily available.   16. To provide a means and method of object recognition and control that is easy to use.   17. To provide a means and method of object recognition and control that is reusable.   18. To provide a means and method of object recognition and control that is quick and effective.   19. To provide a means and method of object recognition and control that is adjustable to varying conditions.   20. To provide a means and method of object recognition and control that is cost effective.   
 
     
    
     
       DRAWING FIGURES 
         [0045]    In the drawings, closely related figures have the same number but different alphabetic suffixes. 
           [0046]      FIG. 1A  through  FIG. 1C  show prior art coded labels, such as bar codes and QR codes. 
           [0047]      FIG. 2  illustrates the system with people with different roles and separate handheld devices communicating with the same database and server. 
           [0048]      FIG. 3A  through  FIG. 3D  illustrate flowcharts of showing the exemplary steps for the methods of the system. 
           [0049]      FIG. 4  illustrates the structure of the database. 
           [0050]      FIG. 5  shows a user with a mobile wireless device with various objects and cameras for remote observation. 
           [0051]      FIG. 6A  through  FIG. 6C  illustrate an example of guided maintenance using coded labels. 
           [0052]      FIG. 7A  through  FIG. 7C  illustrate an example of guided maintenance without using coded labels, using object recognition instead. 
           [0000]    
         
           
                 
               
                 
                 
                 
                 
               
             
                 
                     
                 
                 
                   REFERENCE NUMERALS IN DRAWINGS 
                 
                 
                     
                 
               
               
                 
                     
                 
               
            
             
                 
                     
                   100 
                   a-d 
                   coded label 
                 
                 
                     
                   101 
                     
                   universal product code (UPC) 
                 
                 
                     
                   102 
                     
                   quick response (QR) code 
                 
                 
                     
                   103 
                     
                   high capacity color barcode (HCCB) 
                 
                 
                     
                   120 
                   a-c 
                   scan label 
                 
                 
                     
                   200 
                   a-c 
                   object 
                 
                 
                     
                   210 
                   a-c 
                   handheld device 
                 
                 
                     
                   211 
                   a-c 
                   image data 
                 
                 
                     
                   212 
                   a-c 
                   outgoing database information 
                 
                 
                     
                   213 
                   a-c 
                   incoming database information 
                 
                 
                     
                   220 
                     
                   maintainer 
                 
                 
                     
                   230 
                     
                   user 
                 
                 
                     
                   240 
                     
                   observer 
                 
                 
                     
                   250 
                     
                   database 
                 
                 
                     
                   300 
                     
                   start 
                 
                 
                     
                   301 
                     
                   path 
                 
                 
                     
                   302 
                     
                   capture image 
                 
                 
                     
                   303 
                     
                   path 
                 
                 
                     
                   304 
                     
                   interpret image 
                 
                 
                     
                   305 
                     
                   path 
                 
                 
                     
                   306 
                     
                   determine object 
                 
                 
                     
                   307 
                     
                   path 
                 
                 
                     
                   308 
                     
                   determine person 
                 
                 
                     
                   309 
                     
                   path 
                 
                 
                     
                   310 
                     
                   is person authorized 
                 
                 
                     
                   311 
                     
                   path 
                 
                 
                     
                   312 
                     
                   determine role 
                 
                 
                     
                   313 
                     
                   path 
                 
                 
                     
                   314 
                     
                   no access 
                 
                 
                     
                   315 
                     
                   path 
                 
                 
                     
                   316 
                     
                   maintenance? 
                 
                 
                     
                   317 
                     
                   path 
                 
                 
                     
                   318 
                     
                   perform maintenance 
                 
                 
                     
                   319 
                     
                   path 
                 
                 
                     
                   320 
                     
                   use? 
                 
                 
                     
                   321 
                     
                   path 
                 
                 
                     
                   322 
                     
                   perform use 
                 
                 
                     
                   323 
                     
                   path 
                 
                 
                     
                   324 
                     
                   observe? 
                 
                 
                     
                   325 
                     
                   path 
                 
                 
                     
                   326 
                     
                   perform observation 
                 
                 
                     
                   327 
                     
                   path 
                 
                 
                     
                   329 
                     
                   path 
                 
                 
                     
                   331 
                     
                   path 
                 
                 
                     
                   332 
                     
                   identify part 
                 
                 
                     
                   333  
                     
                   path 
                 
                 
                     
                   334  
                     
                   identify tools 
                 
                 
                     
                   335  
                     
                   path 
                 
                 
                     
                   336  
                     
                   provide instruction 
                 
                 
                     
                   337 
                     
                   path 
                 
                 
                     
                   338 
                     
                   notify observer of start 
                 
                 
                     
                   339 
                     
                   path 
                 
                 
                     
                   340  
                     
                   confirm completion 
                 
                 
                     
                   341 
                     
                   path 
                 
                 
                     
                   342 
                     
                   record completion 
                 
                 
                     
                   343 
                     
                   path 
                 
                 
                     
                   344  
                     
                   notify observer of completion 
                 
                 
                     
                   345 
                     
                   path 
                 
                 
                     
                   346  
                     
                   more tasks? 
                 
                 
                     
                   347 
                     
                   path 
                 
                 
                     
                   348 
                     
                   exit 
                 
                 
                     
                   349 
                     
                   path 
                 
                 
                     
                   350 
                     
                   path 
                 
                 
                     
                   351  
                     
                   path 
                 
                 
                     
                   352 
                     
                   tools required? 
                 
                 
                     
                   353  
                     
                   path 
                 
                 
                     
                   354 
                     
                   identify tools 
                 
                 
                     
                   355  
                     
                   path 
                 
                 
                     
                   356  
                     
                   provide instruction 
                 
                 
                     
                   357 
                     
                   path 
                 
                 
                     
                   358 
                     
                   notify observer of start 
                 
                 
                     
                   359 
                     
                   path 
                 
                 
                     
                   360 
                     
                   confirm use 
                 
                 
                     
                   361 
                     
                   path 
                 
                 
                     
                   362 
                     
                   record use 
                 
                 
                     
                   363 
                     
                   path 
                 
                 
                     
                   364 
                     
                   notify observer of completion 
                 
                 
                     
                   365  
                     
                   path 
                 
                 
                     
                   366  
                     
                   more tasks? 
                 
                 
                     
                   367  
                     
                   path 
                 
                 
                     
                   368 
                     
                   exit 
                 
                 
                     
                   369 
                     
                   path 
                 
                 
                     
                   370 
                     
                   path 
                 
                 
                     
                   371 
                     
                   path 
                 
                 
                     
                   372 
                     
                   instruction needed? 
                 
                 
                     
                   373  
                     
                   path 
                 
                 
                     
                   374 
                     
                   provide instruction 
                 
                 
                     
                   375  
                     
                   path 
                 
                 
                     
                   376  
                     
                   record observation start 
                 
                 
                     
                   377  
                     
                   path 
                 
                 
                     
                   378  
                     
                   communicate with others 
                 
                 
                     
                   379 
                     
                   path 
                 
                 
                     
                   380 
                     
                   record observation finish 
                 
                 
                     
                   381 
                     
                   path 
                 
                 
                     
                   382 
                     
                   more tasks? 
                 
                 
                     
                   383  
                     
                   path 
                 
                 
                     
                   384  
                     
                   exit 
                 
                 
                     
                   385 
                     
                   path 
                 
                 
                     
                   400 
                     
                   roles 
                 
                 
                     
                   402 
                     
                   people 
                 
                 
                     
                   404  
                     
                   objects (tools) 
                 
                 
                     
                   406 
                     
                   tasks 
                 
                 
                     
                   408 
                     
                   task list 
                 
                 
                     
                   410  
                     
                   devices 
                 
                 
                     
                   510  
                     
                   tool 
                 
                 
                     
                   520  
                     
                   mounted camera 
                 
                 
                     
                   530  
                     
                   satellite camera 
                 
                 
                     
                   540  
                     
                   overhead camera 
                 
                 
                     
                   550  
                     
                   mounting structure 
                 
                 
                     
                   560 
                     
                   drone 
                 
                 
                     
                   570 
                     
                   satellite 
                 
                 
                     
                   610  
                     
                   box assembly 
                 
                 
                     
                   620 
                     
                   door 
                 
                 
                     
                   630  
                     
                   door handle 
                 
                 
                     
                   640  
                     
                   outer part 
                 
                 
                     
                   650  
                     
                   inner part 
                 
                 
                     
                     
                 
               
            
           
         
       
       
    
    
     DESCRIPTION OF THE INVENTION 
     Prior Art Coded Labels 
       [0053]      FIG. 1A  through  FIG. 1C  show prior art coded labels  100 . 
         [0054]      FIG. 1A  illustrates an example of a coded label  100  with a barcode, in particular a universal product code (UPC)  101 . A barcode or UPC typically represents a number. The UPC system allows each product to have a unique code making the system universal. However, individual products can only be identified with an additional label such a serial number barcode. 
         [0055]      FIG. 1B  illustrates an example of a coded label  100  with a quick response (QR) code  102 . A QR code  102  has high data density. It can encode free form text and is typically used to encode a direct address of other information such as an URL. 
         [0056]      FIG. 1C  illustrates an example of a coded label  100  with a high capacity color barcode (HCCB)  103 . A HCCB may also contain a direct address such as an URL. 
       Handheld Devices Used for Object Recognition and Control and Instruction 
       [0057]      FIG. 2  illustrates a system with people with different roles and separate handheld devices communicating with the same database and server. Each person is shown with a handheld device  210 . The handheld device  210  is preferably a mobile wireless device. The handheld device  210  includes a camera, which can scan an image of an object  200 . A coded label  100  on object  200  is initially scanned as image data  211   a - c  by any of the handheld devices  210   a - c,  which is operated by a maintainer  220 , a user  230 , and/or an observer  240 , respectively. The observer  240  can be a manager, supervisor, or team leader, etc. The handheld devices  210   a - c  each interacts with the same database  250  to send and receive information. Information sent to the database  250  is outgoing database information  212   a - c  respectively. For example, when maintainer  220  scans the coded label  100  on object  200 , the image data  211   a  is interpreted by the handheld device  210   a  and the decoded data is sent to the database  250  as outgoing database information  212   a.  The maintainer  220  may also be recognized, for example, by scanning scan label  120   a.  Information sent from the database  250  to the handheld devices  210   a - c  is incoming database information  213   a - c,  respectively. Incoming database information  213  may be directed to the maintainer  220 , the user  230 , and/or the observer  240 . The information in database  250  can include instructions, tools list and maintenance logs. The data in the database  250  may describe each person&#39;s role, authorization, security access or other information. For example, after receiving the decoded data from the handheld device  210   a,  incoming database information  213   a  sent to the maintainer  220  would include instructions and a list of tasks, which may be customized to the specific maintainer  220  (as identified e.g. by scan label  120   a ). The maintainer  220  may receive information about the parts on or in the identified object  200 . The maintainer  220  may also receive instructions on how to perform the maintenance and the tools necessary. The maintainer  220  may be shown a video series on the handheld device  210   a  showing each of the steps of the required maintenance. A local handheld device (e.g.  210   a ) may detect whether or not the proper steps were performed. The observer  240  may be in a remote location, and may receive information about the maintainer  220 , or the user  230 , information on progress updates, and visual data (pictures and/or video) to oversee the tasks being performed. 
         [0058]    The communication paths for the outgoing database information  212   a - c  and the incoming database information  213   a - c  are preferably wireless, such as WiFi or 3G or 4G wireless network protocols. 
         [0059]    The mobile wireless handheld devices  210   a - c  can communicate directly to each other as enabled by the database  250 , or may communicate using the database  250  as a teleconferencing server. 
         [0060]    In some embodiments, the observer  240  observes using conventional computer or video conferencing equipment, rather than using a handheld device  210   c.    
       Exemplary Operation 
       [0061]      FIG. 3A  through  FIG. 3D  provide flowcharts for method aspects of an embodiment of the invention. Other embodiments may omit or add steps, or perform steps in different orders. 
         [0062]      FIG. 3A  is a top-level flowchart. The process starts at step  300  and moves along path  301  to a capture image step  302  where the image is captured by a handheld device  210 . The process moves along path  303  to an interpret image step  304  where the image is interpreted. The image may be interpreted by the handheld device  210  or by the database  250 . Then the process moves along path  305  to a determine object step  306  where the object is determined from data stored in the database based on the image data  211 . Then the process moves along path  307  to a determine person step  308 . The person may be determined based on a device code unique to the handheld device (e.g.  210   a  or  210   b ), by the scan label (e.g.  120   a  or  120   b ), or by a face, fingerprint, or iris scan. This checks the identity of the individual running the handheld device  210 . The process then moves along path  309  to a decision  310  where it checks if the person is authorized for object. If not, the process moves along path  313  to “No Access” step  314  where it will send a message stating “No Access” and moves along path  329  back to path  301 . If yes, the process moves along path  311  to a determine role step  312  where the role will be determined for the person determined in step  308 . Then the process decides which sub-process to perform based on the role: maintenance, use, or observation. First the process moves along path  315  to maintenance? decision  316  to decide if the role is maintenance. If yes, the process moves along path  317  to perform maintenance step  318  (see  FIG. 3B ) and then moves along path  327  back to path  301 . If not, the process moves along path  319  to use? decision  320  to decide if the role is use. If yes, the process moves along path  321  to perform use step  322  (see  FIG. 3C ) and then moves along path  327  back to path  301 . If not, the process moves along path  323  to observe? decision  324  to decide if the role is observe. If not the process moves along path  327  back to path  301 . If yes, the process moves along path  325  to perform observation step  326  (see  FIG. 3D ) and then moves along path  327  back to path  301 . 
         [0063]    In some embodiments, the perform observation step  326  is triggered independently, for example, by the notify observer of the start of maintenance message  338  in  FIG. 3B  or by the notify observer of the start of use message  358  in  FIG. 3C . In those embodiments, the observer  240  does not need to scan the object&#39;s label, but instead may be optionally identified by their handheld device  210   c  or by scanning their ID scan label  120   c,  and associated with the observer role by data stored in the database. In other embodiments the observer  240  is identified by the conventional computer or video conferencing equipment they are using or by their physical location. 
         [0064]      FIG. 3B  is a flowchart for the perform maintenance  318  process. The process begins along path  331  to an identify part step  332  where a maintenance part is determined. Next the process moves along path  333  to an identify tools step  334 , where the tools associated with the current maintenance part are identified. The process moves along path  335  to a provide instruction step  336 . Instructional information regarding the current maintenance part, use of tools, how the maintenance is performed is provided, for example, as incoming database information  213   a  from the database  250  to the handheld device  210   a.  The instructional information could include videos of the maintenance process. At event  337 , a notify observer of the start of maintenance message  338  is sent. The handheld device  210   c  associated with the observer  240  is notified along communications path  213   c.  The observer can observe the maintenance procedure via pictures, audio, and/or video transmitted to the handheld device  210   c  (or in some embodiments, to conventional computing or video conferencing equipment). The information in message  338  is sufficient to automatically authorize and enable the observer  240  to observe the maintenance without requiring further actions by the observer  240 . The observer  240  can watch what is happening and will subsequently wait for confirmation of the maintenance completion. When the maintenance is completed, the process moves along path  339  to the confirm completion step  340 , the handheld device  210   a  will send completion information back to the database  250 . The process will then move along path  341  to a record completion step  342  where the completion of the task is recorded in the database  250 . The database  250  can log the time and date of the maintenance as well as logging the person who performed the maintenance, the tools used, how long it took, and so forth. At event  343 , a notify observer of the completion of maintenance message  344  is sent. The observer  240  could verify that the maintenance was performed as required and can then turn attention to other matters. The optional verification could include sending a verification message to the database  250 . Then the process moves along path  345  to more tasks? decision  346  to determine if there are more maintenance tasks. If so, the process moves along path  349  to path  331  to repeat the cycle. If not, the process moves along path  347  to exit  348 . 
         [0065]      FIG. 3C  is a flowchart for the perform use  322  process. The process begins along path  350  to a tools required? decision  352 . If so, the process moves along path  353  to an identify tools step  354  where tools are identified and continues along path  355 . If not, the process moves along path  351  and skips step  354 . At a provide instruction step  356 , information regarding use, and tools if applicable, is provided to the handheld device  210   b  of the user  230 . The instruction could include a checklist, pictures, audio, and/or video. At event  357 , a notify observer of the start of use message  358  is sent. The handheld device  210   c  associated with the observer  240  is notified along communications path  213   c.  The observer  240  can observe the use via pictures, audio, and/or video transmitted to the handheld device  210   c  (or in some embodiments, to conventional computing or video conferencing equipment). The observer  240  may wait for notice of use completion. When use is completed, the process moves along path  359  to the confirm use step  360 , the handheld device  210   b  will send use information back to the database  250 . The process will then move along path  361  to a record use step  362 . The database  250  can log the time and date of the use as well as logging the person who performed the use, the tools used, how long the object was used, and so forth. At event  363 , a notify the observer of the completion of use message  364  is sent. Then the process moves along path  365  to more tasks? decision  366  to determine if there are more use tasks. If so, the process moves along path  369  to path  350  to repeat the cycle. If not, the process moves along path  367  to exit  368 . 
         [0066]      FIG. 3D  is a flowchart for the perform observation  326  process. The process begins along path  370  to an instruction needed? decision  372 . If so, the process moves along path  373  to a provide instruction step  374 , where instruction regarding the observation is provided, and continues along path  375 . If not, the process moves along path  371  and skips step  374 . At the provide instruction step  374 , information is provided to the handheld device  210   c  of the observer  240 . The instruction could include a checklist, pictures, audio, and/or video. In record observation start step  376 , it is noted that the observer  240  has started the observation. During the observations, at event  377 , the observer  240  may communication with others, e.g. the maintainer  220  or the user  230 . The communication with others  378  could be instructional, corrective or supportive information. The handheld device  210   c  associated with the observer  240  will send communication  378  to other handheld devices (e.g.  210   a  or  210   b ). When the observer  240  stops observing, the process moves along path  379  to the record observation finish step  380 , the handheld device  210   c  will send observation information back to the database  250 . The database  250  can log the time and date of the observation as well as logging the person who observed, how long the observation took, the types of communication that occurred during the observation, and so forth. Then the process moves along path  381  to more tasks? decision  382  to determine if there are more observation tasks. If so, the process moves along path  385  to path  370  to repeat the cycle. If not, the process moves along path  383  to exit  384 . 
       Database 
       [0067]      FIG. 4  illustrates the types of information that the database  250  will hold and maintain. The database  250  can be any data repository including relational, hierarchical, etc. The data schema will support various entities including roles  400 , people  402 , objects (including tools)  404 , tasks  406 , task lists  408 , and devices  410 . The storage of this information on a single, central accessible database is valuable to proper communication and coordination. 
         [0068]    The data regarding roles  400  include the roles of various people associated with various objects. For example, an individual may be a maintainer of a particular object, such a machine, but may be a user of another object, such a tool used to fix the machine. The individual may also be a supervisor for another maintainer and will be notified as an observer whenever the maintainer they supervise is performing maintenance. Thus the same individual may have the role of maintainer, user, and observer. These relationships are tracked in the Roles  400 , People  402 , and Objects (Tools)  404  data tables. 
         [0069]    Means for identifying people such as ID badges, fingerprints, face scans, or iris scans are stored in the People  402  data table. Relationships of people  402  to roles  400 , objects  404 , task lists  408 , and devices  410  are also tracked in the database. 
         [0070]    Object specific tasks  406  and instructions associated with those tasks are stored in the Tasks  406  portion of the database. Object specific instructions may include documents, checklists, pictures, audio, and/or video. 
       Exemplary Use 
       [0071]      FIG. 5  shows how a user  230  can use a handheld device  210  and various communication methods. The user  230 , identified by the scan label  120 , can access information from the handheld device  210 . A remote observer  240  can observe using various cameras, such as mounted camera  520  shown on a mounting structure  550 , shown as a pole, an overhead camera  540  shown mounted on an airplane or drone  560 , or even a satellite camera  530  mounted on a satellite  570 . The handheld device  210  can identify object  200  by reading coded label  100  to review maintenance history, to determine what tasks are to be performed using the object  200 , and to get more information regarding the object  200 . The database  250  can assimilate this information and determine whether the user  230  will need tools such as tool  510  for a particular use (or maintenance task). At the same time, the system tracks that the specific user  230  is interacting with the object  200 . The system can notify one or more observers  240  who are required to observe or who may be interested and authorized to observe. The system may automatically observe. For example, if tool  510  is required, the system can notify the user  230 , provide instructional information, including video information to the user  230  via the handheld device  210 , and the monitor the use to ensure that it is performed properly. If the use (or maintenance task) is not performed properly, the system, or an observer  240 , can correct the situation while the user  230  is on site and performing the task. Further, if there is a problem, additional instruction may be provided via pictures, audio, and/or video. Details of the use or maintenance, and optional observation, are stored in the database as part of the maintenance or use history and can be used to improve procedures, training, and safety. 
         [0072]    In some embodiments, the mounted camera  520  (or even a satellite camera  530 ) can scan all of the coded labels  100  and scan labels  120  as well as recognize objects  200  and tools  510 . Once a user  230  and the object  200  are identified in proximity, the system could instruct the user  230  via the handheld device  210  to perform the procedure. The system could observe the entire interaction and provide instruction prior to each task as well as providing corrective feedback and instruction when the wrong actions or parts are observed. The system would also record information regarding the user  230 , the object  200 , and the interactions. 
       Guided Maintenance 
       [0073]      FIG. 6A  through  FIG. 6C  illustrate an example of guided maintenance using coded labels. 
         [0074]      FIG. 6A  shows a box assembly  610 . It has a coded label  100   a  for identification on the upper right-hand corner of the door  620 . The assembly door  620  can be opened by turning the door handle  630 . In this example, the handheld device  210  would image the object, box assembly  610 , including the coded label  100   a.  The system would determine the proper maintenance procedure for the particular box assembly  610 . The system would also confirm that status of the box assembly  610 , e.g. that the door  620  is closed. 
         [0075]    At this point, the handheld device  210  would provide the maintainer  220  with an overview of the maintenance procedure that is required. This instruction could include a checklist, pictures, audio, and/or video. In this example, the instructions would show how to open the door. 
         [0076]      FIG. 6B  shows the box assembly  610  with its door  620  opened. The door handle  630  has also been turned. The opened door  620  with coded label  100   a  exposes another coded label  100   b  on the body of the box assembly  610  and another outer part  640 . The coded label  100   c  identifies the outer part  640 , while the coded label  100   b  is used to help the handheld device recognize that the door  620  is opened. 
         [0077]    At this point, the handheld device  210  recognizes that the door has been properly opened, that the outer part  640  is the expected internal part. Instructions are then provided for the next step, e.g. removing the outer part  640 . 
         [0078]      FIG. 6C  shows a box assembly  610  as in  FIG. 6B  with its door  620  opened. The outer part  640  with coded label  100   c  has been removed exposing the inner part  650  with coded label  100   d.  The outer part  640  has been placed along side the box assembly  610 . The inner part  650  has a coded label  100   d  to not only identify the inner part  650 , but also to help the handheld device recognize that the outer part  640  has been removed. 
         [0079]    At this point, the handheld device  210  recognizes that the outer part  640  has been properly removed, that the inner part  650  is the expected part. Instructions are then provided for the next step, e.g. replacing the inner part  650 . When the inner part  650  has been replaced, the details of the replacement are recorded in the database  250 . The system can also track who did the replacement, how long it took, and any problems encountered. 
         [0080]    The system would also provide instruction and tracking of the reassembly of the box assembly. 
         [0081]    This example has been exemplary but the same principles could apply to any piece of equipment. 
         [0082]      FIG. 7A  through  FIG. 7C  illustrate an example of guided maintenance without using coded labels. 
         [0083]    All the parts in  FIG. 7A through 7C  for the box assembly  610  and the maintenance are identical to the ones described in  FIG. 6A through 6C  with the exception of not having coded labels  100   a - d.  In this embodiment, the invention would identify all of the parts without the use of coded labels  100 . Instead the handheld device  210  would use object recognition to determine the status of the maintenance process and to determine which parts were visible at any point in the process. The database  250  could store images showing what is expected at each point in the procedure and the system can compare what is being scanned by the cameras (of the handheld device  210 , or mounted camera  520 ) with what is in the database  250 . 
         [0084]      FIG. 7A  shows a box assembly  610 . The assembly door  620  can be opened by turning the door handle  630 . In this example, the handheld device  210  would image the object, box assembly  610 . The system would determine the proper maintenance procedure for the identified box assembly  610 . The system would also confirm that status of the box assembly  610 , e.g. that the door  620  is closed. 
         [0085]    At this point, the handheld device  210  would provide the maintainer  220  with an overview of the maintenance procedure that is required. This instruction could include a checklist, pictures, audio, and/or video. In this example the instructions would show how to open the door. 
         [0086]      FIG. 7B  shows the box assembly  610  with its door  620  opened. The opened door  620  exposes outer part  640 . 
         [0087]    At this point, the handheld device  210  recognizes that the door has been properly opened, that the outer part  640  is the expected part. Instructions are then provided for the next step, e.g. removing the outer part  640 . 
         [0088]      FIG. 7C  shows a box assembly  610  as in  FIG. 7B  with its door  620  opened. The outer part  640  has been removed exposing the inner part  650 . The outer part  640  has been placed along side the box assembly  610 . 
         [0089]    At this point, the handheld device  210  recognizes that the outer part  640  has been properly removed, that the inner part  650  is the expected part. Instructions are then provided for the next step, e.g. replacing the inner part  650 . When the inner part  650  has been replaced, the details of the replacement are recorded in the database  250 . The system can also track who did the replacement, how long it took, and any problems encountered. 
         [0090]    The system would also provide instruction and tracking of the reassembly of the box assembly. 
       Advantages 
     Universality 
       [0091]    The unique aspects of this invention allow for universal usage. Conventional methods due to constraints inherent in their design, or due to historical circumstances, are unable to be used universally. Even the Universal Product Code style of barcode is not used for labeling in a variety of industrial, military, postal, or inventory-related applications. 
       Common Repository 
       [0092]    The present invention can be used to provide a common repository for central storage of data that can be shared among users throughout the organization and between organizations, which are working together. 
       Role Based 
       [0093]    The present invention can be used to assign roles to individuals associated with various objects or tasks. 
       Discipline Based 
       [0094]    The present invention can be used to assign disciplines to individuals and provide instruction, tracking and control based on those disciplines. 
       Permission Based 
       [0095]    The present invention can be used to assign permissions based on roles. 
       Specific Media Rich Instructions 
       [0096]    The present invention can be used to provide instructions regarding specific objects to specific individuals in a variety of mediums including documents, checklists, pictures, audio, and/or video. 
       Verification and Ratification 
       [0097]    The present invention can be used to verify and ratify information for objects and individuals. 
       Maintenance Notification and Instructions 
       [0098]    The present invention can be used to provide maintenance notification and instructions regarding specific objects to specific individuals. 
       Log Maintenance And Events 
       [0099]    The present invention can be used to log maintenance and events for objects. 
       Track the Life of an Object 
       [0100]    The present invention can be used to track a variety of events for objects. 
       Reporting 
       [0101]    The present invention can be used to provide reports on given objects and individuals or a combination of the two. 
       Presenting Information in a Variety of Mediums 
       [0102]    The present invention can be used to provide information in a variety of mediums including documents, checklists, pictures, diagrams, audio and/or video. 
       Problem Solving 
       [0103]    The present invention can be used to provide the ability for problem solving via knowledge bases that can be associated with objects and individuals. 
       Inexpensive and Readily Available 
       [0104]    The present invention can be used as a defacto standard for object identification making it inexpensive and readily available. 
       Easy to Use 
       [0105]    The present invention is easy to use. 
       Inexpensive 
       [0106]    Conventional methods are expensive, requiring the user to invest in expensive code-reading technology and printing costs. The present invention is inexpensive. 
       Quick 
       [0107]    The present invention&#39;s unique design allow for quick use, time-saving when compared to conventional methods. 
       Effective 
       [0108]    The present invention is effective, allowing for multiple sources of identification and providing helpful information to streamline the work that needs to be done. 
       Conclusion, Ramification, and Scope 
       [0109]    Accordingly, the reader will see that the improved, universal, role and discipline based, low cost means and method of object identification, tracking and control are easy to use, quicker and accurate and can be used with objects or individuals to provide up-to-date pertinent information using methods such as on screen instructions, checklists, videos, pictures, diagrams, etc. or any combination thereof. 
         [0110]    While the above descriptions contain several specifics these should not be construed as limitations on the scope of the invention, but rather as examples of some of the preferred embodiments thereof. Many other variations are possible. The variations could be used without departing from the scope and spirit of the novel features of the present invention. 
         [0111]    Accordingly, the scope of the invention should be determined not by the illustrated embodiments, but by the appended claims and their legal equivalents.