Abstract:
A method and system for remote assistance and review of a technician or multiple technicians, in real time, working with equipment of various complexity. A technician or multiple technicians at a remote location are coupled by a wireless means to an advisor at a local station, so that the advisor may view and hear the same stimuli as the technician, that the advisor and technician may communicate. The technician has limited training or otherwise in need of support, and may be a field engineer, technician or maintenance personnel. The advisor has extensive training and able to provide technical support, and generally has extended and specialized knowledge with regard to the remote apparatus, and may be a technical expert on the remote apparatus. The technician may comprise an individual or group with technical training and knowledge, but lacking managerial or other authority, while the advisor comprises an individual or group with such authority. The technician communicates with the advisor by visual cues or ordinary speech, while the advisor views and listens to the remote apparatus. The advisor gives advice to the technician for manipulating or repairing the remote apparatus. Alternatively, an intermediate advisor may advise the technician and be advised by a higher-level advisor.

Description:
CROSS-REFERENCE TO RELATED APPLICATION  
       [0001]    This Application is a Continuation of U.S. application Ser. 09/175,112 filed Oct. 19, 1998. 
     
    
     
       BACKGROUND OF THE INVENTION  
         [0002]    1. Field of the Invention  
           [0003]    This invention relates to a wireless video audio data remote system.  
           [0004]    2. Description of Related Art  
           [0005]    The nature of business organizations and their employees is generally such that, in most any subject, there are a relatively small number of persons with extensive training and experience (e.g., “experts” ) and a relatively large number of persons with only limited training and experience (e.g., technicians). This problem is exacerbated by the relatively larger costs associated with the former. Accordingly, when a business organization seeks to apply a person&#39;s skills to a problem, it is often faced with the fact that persons with the extensive training and skill are a scarce resource.  
           [0006]    With many problems, and in particular with the problem of servicing and trouble-shooting equipment, experience shows that most tasks require only a subset of the full experience that characterizes experts in the field, and that those tasks can be adequately carried out by entry-level technicians. However, when a technician is confronted with a task requiring an expert, often the only indicator thereof is the technician&#39;s inability to solve the problem. Sometimes the technician&#39;s lack of extensive knowledge can actually make the problem worse. These effects serve to increase the cost, time, effort, and frustration associated with the servicing and troubleshooting equipment.  
           [0007]    One method in the prior art has been to advise technicians at a fixed location using audio video and data transmission over various wired networks. An example of this is U.S. Pat. No. 5,619,183 (Ziegra et al.). The main disadvantage of that fixed-site system is that remote-site technician oversight is impractical when a mobile work force requires video, audio and data transmission from continually changing job site locations. Another disadvantage is that the fixed-site to fixed-site system software does not fully utilize the idea of “leveraged expertise”.  
           [0008]    Accordingly, it would be advantageous to provide a method and system by which technicians can perform the tasks they are adequately trained for, in a wireless environment, while allowing experts to assist them when expert assistance is required.  
         SUMMARY OF THE INVENTION  
         [0009]    The invention provides a method and system for remote assistance and review of a technician or group of technicians working with equipment of various complexity. In a preferred embodiment, a technician at a remote job site is coupled to an advisor manning a local station (where “local” and “remote” are relative to a remote apparatus being controlled or serviced by the technician, and do not necessarily denote large distance) in such a manner that the advisor may view and hear the same stimuli as the technician and that the advisor and technician can communicate.  
           [0010]    In a preferred embodiment, a technician at a remote job site may be coupled by a wireless communication link(s) to a local station. The technician at the remote job site wears an apparatus consisting of a video and audio sensor, such as a camera and a microphone, and a receiver for the communication link, such as earphone or speaker and a wireless portable data processor. The communication link comprises a wireless communication path to/from the local station, and may further comprise data encoding, compression, and error correction/detection devices. The local station comprises a video and audio display, such as a monitor and a speaker, software that allows for real-time communication to multiple technicians, and a transmitter for the communication link with the remote job site, such as a microphone.  
           [0011]    In a preferred embodiment, the technician may comprise an individual or group with limited training or otherwise in need of support, such as a field engineer or technician. The technician(s) should generally know how to operate the remote job site apparatus, but need not have extended or specialized knowledge with regard thereto. The advisor may comprise an individual or group with extensive training and able to provide technical support, who generally does have extended and specialized knowledge with regard to the remote job site apparatus, such as a technical expert on the remote job site apparatus. In an alternative embodiment, the technician(s) may comprise an individual or group with technical training and knowledge, but lacking managerial or other authority, while the advisor(s) are an individual or group with such authority.  
           [0012]    In a preferred embodiment, the technician(s) couples the remote job site communication apparatus to the wireless communication link(s) and to the local station communication apparatus. The technician(s) may communicate with the advisor(s) by visual cues or ordinary speech, while the advisor(s) may view and listen to the remote job site apparatus. The advisor(s) may give advice to the operator for manipulating the remote job site apparatus, and may manipulate the remote job site apparatus directly by means of the control signal or data signal feeds. Thus, the technician(s) may service/repair/operate the remote job site apparatus as if the advisor were peeking over his shoulder.  
           [0013]    In an alternative embodiment, an intermediate advisor may advise/control the technician(s) and be advised/controlled by a higher-level advisor. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0014]    [0014]FIG. 1 shows a block diagram of a wireless remote system.  
         [0015]    [0015]FIG. 2 shows a block diagram of wireless portable communication apparatus for a remote system.  
         [0016]    [0016]FIG. 3 shows a block diagram of a local station for a wireless remote system.  
         [0017]    [0017]FIG. 4 shows a block diagram of a vehicular based portable processor for a remote system.  
         [0018]    [0018]FIG. 5 shows a block diagram of a centralized expertise station coupled to multiple wireless portable remote systems.  
         [0019]    [0019]FIG. 6 shows a flow diagram of a method for operating a wireless remote system.  
     
    
     DESCRIPTION OF THE PREFERRED EMBODIMENT  
       [0020]    In the following description, a preferred embodiment of the invention is described with regard to preferred process steps and data structures. However, those skilled in the art would recognize, after perusal of this application, that embodiments of the invention may be implemented using a set of general purpose computers operating under program control, and that modification of available general purpose computers to implement the process steps and data structures described herein would not require undue invention.  
       General Features of the Method and System  
       [0021]    [0021]FIG. 1 shows a block diagram of a wireless remote system.  
         [0022]    A wireless remote system  101  comprises a mobile field technician  102  at a remote job site utilizing a wireless portable processor  103  and a wireless audio headphone/microphone  104  and a wireless camera  105  coupled by a wireless network  106  to the local station  108  and being advised by the local master technician  109 .  
         [0023]    The local station  108  is comprised of a local processor  110  and the local master technician audio/microphone headset  111  utilized by the local master technician  109 .  
         [0024]    As described herein, the mobile field technician  102  may operate/service/maintain an apparatus at a remote job site with the advice and control of the local master technician  109 :  
         [0025]    (a) The local master technician  109  may view and hear the same stimuli at the remote site as the mobile field technician  102 , by means of audio and video sensors at the remote job site.  
         [0026]    (b) The mobile field technician  102  may communicate with the local master technician  109  by means of the wireless network  106 .  
         [0027]    (c) The mobile field technician  102  may communicate with the local master technician  109  by means of the wireless network  106  and/or by means of the Internet  107 .  
         [0028]    As used herein, “local” and “remote” are relative to logical control of the remote job site apparatus and do not necessarily denote large distance. For example, the remote job site and the local station  108  may be located in the same building or even in the same room, where it is desired to logically separate the function of the mobile field technician  102  and the local master technician  109  as in a local training environment. Similarly, the mobile field technician  102  need not be physically co-located with the remote job site apparatus where it is desired to control that remote job site apparatus by means of physically distant tools and software.  
         [0029]    In a preferred embodiment, remote air conditioning system control software such as Parker Controls VVT system would allow a mobile field technician  102  to make adjustments to a facility&#39;s hvac system with guidance from the local master technician  109  without either of them physically being at the job site.  
       Remote Wireless System Components  
       [0030]    [0030]FIG. 2 shows a block diagram of the portable components of the wireless remote system. The mobile field technician  102  wears a group of components that comprise a wireless remote system  101 . The wireless remote system  101  is comprised of a wireless portable processor  103 , an earphone headset  104 , a microphone  128 , and a video sensor/camera  105 . In a preferred embodiment, the wireless portable processor  103  is of sufficient size and weight to be worn by the mobile field technician  102  and has an embedded web browser  118  to allow for wireless connection to the Internet  107 , an embedded codec  117  for audio/video data compression, a wireless modem  119 , to connect to the wireless network  106 , a wireless RF video receiver  120 , a wireless RF audio transceiver  116 , a data input jack  114 , to receive data from a testing meter  126 , an audio input jack  129  to alternatively connect to the wireless earphone headset  104 , and a video input jack  115  to alternatively connect to the video sensor/camera  105  and a communication port  132 , to alternatively connect the wireless portable processor  103  to a cellular phone  112 , that would alternatively couple the wireless remote system  101  to the wireless network  106 .  
         [0031]    In a preferred embodiment, the wireless earphone headset  104  comprises a set of headgear or a helmet suitable for wearing by the mobile field technician  102 , having the microphone  128  and the video sensor/camera  105  attached thereto or embedded therein, in suitable locations relative to the mobile technician&#39;s ears, eyes, and mouth. For example, the wireless earphones  104  are preferably disposed near the mobile field technician&#39;s ears, the wireless camera  105  is preferably disposed near the side of the mobile field technician&#39;s head and pointed ahead at a center of-vision line relative to the operator&#39;s head, and the microphone  128  is preferably disposed near the mobile field technician&#39;s mouth. Alternatively, the wireless camera  105  may be detached from the headset  104  and placed at a distance from the mobile field technician  102  in such a manner as to provide the local master technician  109  a larger field of view while mounted to a stationary platform.  
         [0032]    In a preferred embodiment, the wireless earphone headset  104  would comprise a microphone  128  for voice communication with the local master technician  109 , an earphone  133  for use on one ear by the mobile field technician  102 , and a wireless transceiver  130  to couple with the wireless headphone transceiver  116 . Alternatively, the earphone headset  104  could be coupled to the wireless portable processor  103  by means of wired connection to the audio imput jack  129 .  
         [0033]    In a preferred embodiment, the wireless earphone headset  104  comprises the NCHSM-776 product, by Specialty Products, Inc., P.O. box 30665, Reno, Tex. 75462, and the GRT-1001 transceiver product, by Polaris Industries, Inc., 470 Armour Drive, Atlanta Ga. The NCHSM-776 product is a combination earphone headset  104  and a noise canceling microphone  128 . In an alternative embodiment, the wireless earphone headset would have a jack for a wired connection to the wireless portable processor  103 . In an alternative embodiment, the wireless earphone headset  104  would be coupled to a cellular phone  112  and operate independently of the wireless portable processor  103 .  
         [0034]    In a preferred embodiment, the video/sensor camera  105  would be detachable from the wireless earphone headset  104  and would comprise a camera  105  and a wireless RF video transmitter  131 . The wireless RF video transmitter  131  would be wirelessly coupled to an RF receiver  120  that would be mechanically coupled, or embedded in, the wireless portable processor  103 . Alternatively, the camera  105  and the wireless earphone headset  104  would use the same transceiver  116  in a dual band mode and thus only need one.  
         [0035]    In a preferred embodiment, the video/sensor camera  105  would comprise the SCI-VC-600 color board camera product, by Synergy Concepts Inc., P.O.box 803089 Dallas Tex. 75380-3088 and a wireless RF video transmitter  131  model GRT1001 by Polaris Industries, Inc., 470 Armour Drive, Atlanta Ga.  
         [0036]    In a preferred embodiment, the camera  105  can be configured to alternatively provide high-resolution still images in place of continuous video images, represented by an analog video signal in a standard format. In a preferred embodiment, the camera  105  would be detachable from the wireless earphone headset  104  and be functionally independent.  
         [0037]    In a preferred embodiment, the wireless portable processor  103  comprise the 220E product hand-held computer, by Hitachi Inc., 2000 Sierra Point Parkway, Brisbane, Calif. 94005-1835.  
         [0038]    In a preferred embodiment, the wireless portable processor  103  would be worn on a belt pack or on a backpack coupled alternatively to an external battery pack that would be worn on a belt pack or a backpack.  
         [0039]    An output of the camera  105  is wirelessly coupled to a local video codec  117 , which digitizes the video signal output by the camera  105 , to produce a stream of digital video data. In a preferred embodiment, the local video codec  117  also compresses the digital video data using a wave let full color motion compression algorithm performed in real time. In a preferred embodiment, the local video codec  117  comprise the HARC-C compression engine product, by Houston Advanced Research Center, Inc., 16800 Greenspark Drive, Ste 140 North, Houston, Tex. 77060. However, in an alternative embodiment, the local video codec  117 , may use another video compression standard, such as the MPEG I or MPEG II compression standard. The MPEG I and or the MPEG II compression standards are known in the art.  
         [0040]    The wireless portable processor  103  utilizes a web browser  118  software to access the Internet  107 . In a preferred embodiment, the web browser  118  would comprise the Netscape 4.0 product, by Netscape Inc., 501 E. Middlefield Rd., Mountain View, Calif. 94043.  
         [0041]    The wireless portable processor  103  is mechanically coupled to or is embedded within, a wireless modem  119 . The wireless modem  119  is coupled by a wireless means to the wireless network  106  which may be coupled to the Internet  107 . The audio/video data from the wireless portable processor  103  can be coupled alternatively via the wireless network  106  and/or the Internet to the local station  108 . In a preferred embodiment, the wireless modem  119  comprises the Type III PCMCIA CDPD modem, by Inot, 1255 W. 15th. St., Plano Tex. 75075-7270. In a preferred embodiment, the wireless modem  119  would be connected to the wireless portable processor via a PCMCIA slot. Alternatively, the wireless portable processor  103  would be coupled to the wireless network  106  via a cabled connection to a cellular phone  112 . In a preferred embodiment, the cellular phone  112  would comprise a model 2160i by Nokia Corp., 6000 Connection Dr., Irving, Tex. 75039.  
       The Local Station  
       [0042]    [0042]FIG. 3 shows a block diagram of a local station  108  for a remote system.  
         [0043]    The local station  108  comprises an operator headset  111 , including earphones  135  and a microphone  134 , a local processor  110 , local station software  136 , a local station data interface  123  and a local master technician  109 .  
         [0044]    In a preferred embodiment, the local master technician  109  would don the operator headset  111  and couple the local processor  110  to the wireless network  106  and/or the Internet  107  via a web browser  118  and the local station data interface  123 .  
         [0045]    In a preferred embodiment, the local station  108  would be coupled by a wireless means to the wireless portable processor  103  and the mobile field technician  102  via the wireless network  106  and/or the Internet  107 .  
         [0046]    In a preferred embodiment, the local master technician  109  would communicate with the mobile field technician  102  via normal speech and visual signals. The local master technician  109  would view the visual signals from the mobile field technicians&#39; camera  105  on the video monitor of the local processor  110  and hear the audio/speech from the mobile field technician  102  using the earphones  135  and give direction to the mobile field technician  102  via the microphone  134 .  
         [0047]    In a preferred embodiment, the local processor  110  would comprise a computer of sufficient capacity to allow for a plurality of audio/video signals from a plurality of mobile field technicians  102  to be processed simultaneously. The local station software  136  would utilize a plurality of video windows that would contain the video signals from the individual mobile field techneicians&#39; cameras  105 . When activated by the local master technician  109 , the individual reduced video windows  122  become enlarged video windows  121  to facilitate observation of details. The mobile field technicians  102  could signal their desire to audibly communicate with the local master technician by an electronic signal, such as a flashing light or icon, within their reduced video windows  122 . The local station software would also track the time each individual mobile field technician  102  was on-line and transmitting data. Where appropriate, the local processor  110  may perform data compression, error detection and correction for audio and video signals transmitted from the mobile field technician  102  via the wireless portable processor  103 .  
         [0048]    In an alternative embodiment, the local station  108  is comprised of a local master technician  109  utilizing a wireless portable processor  103  to communicate with one or more mobile field technicians  102 . The term “local master technician” is not to necessarily imply that the local master technician  109  is constantly in a fixed location, but may himself/herself be mobile. Similarly, the term “mobile field technician” is not to necessary imply that the mobile field technician is continually moving, but may himself/herself, be at a fixed job site for a varying period of time.  
         [0049]    In a preferred embodiment, the local station software would comprise the 3.0 cu-see-me product by Whitepines Inc. of 542 Amherst Street, Naushua, N.H. 03063.  
         [0050]    In a preferred embodiment, the local processor  110  would comprise the PC300GL product by IBM, San Jose Calif.  
         [0051]    In a preferred embodiment, the local processor web browser  118  would comprise the Netscape 4.0 product by Netscape Inc., 501 E. Middlefield Road, Mountain View, Calif. 94043.  
         [0052]    In a preferred embodiment, the operator headset would comprise the SP-NCHSM-776 product by Specialty Products, P.O. box 30665 Reno, Tex. 75462.  
         [0053]    The local station data interface  123  is any connecting point that couples the local processor  110  to the wireless network  106  and/or the Internet  107 . It could be a standard POTS telephone line, an ISDN line, dedicated T 1 , fiber optic, satellite, broad band RF or wireless microwave broad band. It would be clear to those skilled in the art that these different methods would be workable with the disclosure of the present invention, and would not require any undue invention.  
         [0054]    In a preferred embodiment, the local station data interface  123  would comprise a dedicated ISDN line to allow for adequate data transmission to and from the local processor  110  from the wireless network  106 .  
       The Wireless Network  
       [0055]    The wireless network may comprise any wireless link capable of carrying sufficient information between the wireless portable processor  103  and the local processor  110 , such as a CDPD cellar network, TDMA, CDMA, ARDIS, RAM, spread spectrum RF, satellite and/or broad band microwave. In a preferred embodiment, the wireless network  106  comprises the CDPD cellular network coupled to an Internet service provider. However, the wireless network  106  could be utilized as a direct communication link between the wireless portable processor  103  and the local processor  110  without using the Internet  107 . It would be clear to those skilled in the art that these alternative wireless networks would be workable with the disclosed invention, without any undue invention.  
       Vehical Based Portable Processor  
       [0056]    [0056]FIG. 4 shows a block diagram of a vehicle-based wireless remote system.  
         [0057]    The vehicle-based wireless remote system comprises a wireless portable processor  103 , a vehicular mounted wireless network antenna  125 , and a vehicular mounted RF antenna  137 . The mobile field technician  102  would move about a job site while transmitting/receiving audio and video data. The camera  105  and the wireless earphone headset  104  would transmit/receive data via a dual channel audio/video transceiver  138  mounted either on the wireless earphone headset  104 , or alternatively, on a belt or back pack. The dual channel audio/video transceiver  138  would relay the audio/video data by a wireless means to a vehicular mounted RF antenna  137  which would be coupled to a dual channel audio/video transceiver  138  coupled to or embedded with the wireless portable processor  103 . The wireless portable processor  103  would relay the audio/video data from the mobile field technician  102  to the wireless network  106  via the wireless modem  119  that would be coupled to a vehicular mounted wireless network antenna  125 .  
         [0058]    In a preferred embodiment, the dual channel audio/video transceiver  138  would comprise the GFI-1001 model by Polaris Industries, 470 Armour Drive, Atlanta, Ga.  
       Centralized Expertise Station  
       [0059]    [0059]FIG. 5 shows a block diagram of a centralized expertise station  127 .  
         [0060]    A centralized expertise station  127  is comprised of one or more local master technicians  109  and one or more local processors  110 . The local master technicians  109  would guide and advise a plurality of mobile field technicians  102  via a wireless network  106  and/or the Internet  107 . Alternatively, the mobile field technicians  102  need not be technicians at all but may be individuals only in need of the specialized information that the local master technician  109  is providing. Accordingly, the local master technicians  109  may not be technicians at all, but may be individuals with knowledge or knowledge resources sought by others. The centralized expertise station could be a knowledge resource for hire that was made accessible via the wireless remote apparatus  101 . Alternatively, the centralized expertise station  127  could be a no-fee based center for information that was made accessible via the wireless remote apparatus  101  .  
         [0061]    In a preferred embodiment, the centralized expertise station  127  would comprise a plurality of local master technicians  109  who represent a variety of trades and/or have other specialized knowledge that is sought by individuals utilizing wireless portable processors  103 . An individual that uses the wireless portable processor  103  could be a maintenance employee of an apartment project and require oversight and information that would be provided by a local master technician  109  that would allow the maintenance man to perform a repair. In a preferred embodiment, the maintenance personnel would couple to the centralized expertise station  127  via a wireless portable processor  103  coupled to the wireless network by a wireless means  106  and the Internet  107  and receive guidance and information from a local master technician  109  on a fee-for-use or contract basis.  
       Methods of Operation  
       [0062]    [0062]FIG. 6 shows a flow diagram of a method of operating a wireless remote system.  
         [0063]    In a step  601 , the mobile field technician  102  would don the wireless earphone headset  104  and a camera  105 .  
         [0064]    In a step  602 , the mobile field technician  102  would couple the wireless portable processor  103  to the wireless network  106 . In a preferred embodiment, the mobile field technician  102  would couple the wireless portable processor  103  to the cellular network and connect to the Internet  107 .  
         [0065]    In a step  603 , the wireless portable processor  103  transmits audio/video to the local station  108  via the wireless network  106 .  
         [0066]    In a step  604 , the local processor  110  receives the audio/video from the mobile field technician  102 .  
         [0067]    In a step  605 , the local master technician  109  dons the operator headset  111  and views the computer monitor on the local processor  110  and hears and views the audio/video data transmitted from the mobile field technician  102  via the wireless portable processor  103  and the wireless network  106 . In a preferred embodiment, the local master technician  109  would view the video data from the camera  105  in on-screen video windows that can be enlarged for better detail observation and reduced in order to view multiple field technician video windows at the same time. In a preferred embodiment, the local processor  110  would use the White Pines 3.0 video conference software.  
         [0068]    In a step  606 , the local master technician  109  communicates verbally with the mobile field technician  102  using the local station microphone  134  and the local station earphones  135  that comprise the operator headset  111 .  
         [0069]    In a step  607 , the local master technician  109  directs the mobile field technician  102 .  
         [0070]    Although this preferred method of operation is disclosed with regard to a technician and an expert, collectively performing the task of troubleshooting remote apparatus, it would be clear to those skilled in the art, after perusal of this application, that there are many alternative tasks that may be performed by the mobile field technician  102 , with the assistance of the local master technician  109 , using substantially the same method and system. Moreover, it would be clear to those skilled in the art, after perusal of this application, how to modify the system disclosed herein, and known equipment, to implement such alternative tasks without undue invention.  
         [0071]    For example, the following are alternative embodiments of the invention:  
         [0072]    1. The mobile field technician  102  may not be a technician at all but is only an individual in need of information that would be provided via the wireless portable processor  103 .  
         [0073]    2. The local master technician  109  may not be a technician at all but may simply be an information provider via the wireless remote system.  
         [0074]    3. The mobile field technician  102  may not wear or carry the wireless portable processor  103  but would transmit audio/video to a vehicle located processor (FIG. 4) that would relay the audio/video from the wireless earphone headset  104  and the wireless camera  105  to the wireless network  106  via a wireless network antenna  125 .  
         [0075]    4. The mobile field technician  102  may not wear the camera  105  but would alternatively set the camera on a stationary platform in a manner so as to observe the desired area or object. The camera  105  would transmit the video utilizing an attached wireless video transmitter to the wireless portable processor  103  or, alternatively, to a vehicle located processor as shown in FIG. 4.  
         [0076]    5. The mobile field technician  102  may not transmit audio via the wireless portable processor  103  at all but would alternatively transmit/receive audio via a cell phone.  
         [0077]    6. The local station  108  may alternatively be a centralized expertise station  127  where a plurality of master technicians  109  would advise a plurality of mobile field technicians  102 . The local master technicians  109  may represent different trades or areas of expertise and may offer that expertise on a fee or contract basis. Alternatively, the local master technicians  109  may not be technicians at all but simply information providers or allow access to information in a fee based or non fee based environment via the wireless portable processor  103 .  
         [0078]    7. The local master technician  109  may alternatively utilize a wireless portable processor  103  to advise one or more mobile field technicians  102  using wireless portable processors  103 . The local master technician  109  may be at the same location or job site but find it necessary to advise one or more technicians that may be separated from each other or separated from the local master technician  109  or otherwise in need of instruction. An example would be a local master technician  109  advising mobile field technicians that are on a building roof while local master technician may be on the first floor.