Abstract:
A fatigue monitoring system is disclosed for use with a machine. The fatigue monitoring system may have an input device configured to generate a first signal indicative of an activity performed by a machine operator, a scanning device configured to generate a second signal indicative of a recognized characteristic of the machine operator, and a warning device. The fatigue monitoring system may also have a controller in communication with the input device, the scanning device, and the warning device. The controller may be configured to determine a time between generations of the first signal, to make a comparison of the recognized characteristic with a threshold characteristic, and to selectively activate the warning device based on the time or the comparison.

Description:
RELATED APPLICATIONS 
       [0001]    This application is based on and claims priority to U.S. Provisional Application No. 62/053,164 filed on Sep. 21, 2014, the contents of which are expressly incorporated herein by reference. 
     
    
     TECHNICAL FIELD 
       [0002]    The present disclosure relates generally to an operator fatigue monitor system and, more particularly, to a system for monitoring fatigue in operators of a machine such as a train locomotive 
       BACKGROUND 
       [0003]    Fatigue is a known problem in the transportation industry. Operators can become fatigued after extended periods of time of machine control, Some machines require repetitive movements from the operator that can be tiring, while other machines require very little operator movement that can result in drowsiness. In either situation., the operator may lose some ability to concentrate and the operator&#39;s attentiveness may dull, This can result in improper machine control, delayed machine control, or lack of machine control. Any of these results can cause a loss in productivity, profitability, and safety. 
         [0004]    One exemplary system for helping an operator stay alert is commonly known as a vigilance control device. A vigilance control device includes lights and sounders that are connected to timers. The operator is required to provide some type of input with a minimum frequency that is controlled by the timers. If the input is not received within a particular time period, a warning in the form of a flashing light or a sounding siren is initiated, If the operator still does not provide any input, the vigilance control device automatically implements a penalty in the form of a machine speed reduction or activation of the machine&#39;s brakes. Such a system is described in a publication by the Man Vehicle Laboratory of the Massachusetts Institute of Technology titled “Locomotive In-Cab Alerter Technology Assessment” that published on Nov. 20, 2006 (the “Alerter Publication”). 
         [0005]    In addition to describing conventional activity-based Alerters, the Alerter Publication also describes a new type of Alerter that automatically detects operator fatigue without operator input. In particular, the automatic Alerter relies on video images of the operator, and uses computer machine vision algorithms to identify specific features in the video images (e.g., head pose, gaze direction, and eyelid position). Based on these features, the automatic Alerter then generates the warnings and/or penalty described above. 
         [0006]    While the conventional Alerter and the automatic Alerter may both provide some benefit in keeping an operator vigilant and alert, they both suffer drawbacks. For example, the conventional Alerter has been determined to encourage preemptive behavior that avoids activation of startling warnings. In addition, it has been found that train operators can, over time, become accustomed to providing input to the Alerter as a reflexive activity that does not require concentration and that can even be done while asleep. Further, the automatic Alerter can sometimes misread an operator&#39;s face, particular when lighting conditions are poor, when the operator is wearing glasses, or during other similar conditions. 
         [0007]    The present disclosure is directed at overcoming one or more of the shortcomings set forth above and/or other problems of the prior art. 
       SUMMARY 
       [0008]    In one aspect, the present disclosure is directed to fatigue monitoring system for a machine. The fatigue monitoring system may include an input device configured to generate a first signal indicative of an activity performed by a machine operator, a scanning device configured to generate a second signal indicative of a recognized characteristic of the machine operator, and a warning device. The fatigue monitoring system may also include a controller in communication with the input device, the scanning device, and the warning device. The controller may be configured to determine a time between generations of the first signal, to make a comparison of the recognized characteristic with a threshold characteristic, and to selectively activate the warning device based on the time or the comparison. 
         [0009]    In another aspect, the present disclosure is directed to another fatigue monitoring system for a machine. This fatigue monitoring system may include an input device configured to generate a first signal indicative of an activity performed by a machine operator, a scanning device configured to generate a second signal indicative of a recognized characteristic of the machine operator, a warning device, and a controller in communication with the input device, the scanning device, and the warning device. The controller may be configured to determine a time between generations of the first signal, to selectively implement a first level response when the time between generations of the first signal exceeds a first threshold time, and to selectively implement a second level response when the time between generations of the first signal exceeds a second threshold time greater than the first threshold time. The controller may also be configured to make a comparison of the recognized characteristic with a threshold characteristic, and to selectively implement the second level response based on the comparison. 
         [0010]    In yet another aspect, the present disclosure is directed to a method of monitoring fatigue in a machine operator. The method may include receiving inputs indicative of an activity performed by the machine operator, and scanning a facial characteristic of the machine operator. The method may also include determining a time between receipt of the inputs, and making a comparison of the characteristic with a threshold characteristic. The method may further include selectively activating a warning device based on the time or the comparison. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWING 
         [0011]      FIG. 1  is an isometric illustration of an exemplary disclosed locomotive; 
           [0012]      FIG. 2  is an isometric illustration of an exemplary disclosed cabin that may form a portion of the locomotive of  FIG. 1 ; 
           [0013]      FIG. 3  is a diagrammatic illustration of an exemplary disclosed fatigue monitoring system that may be used in conjunction with the locomotive and cabin of  FIGS. 1 and 2 ; and 
           [0014]      FIG. 4  is a flowchart depicting an exemplary disclosed method that may be performed by the fatigue monitoring system of  FIG. 3 . 
       
    
    
     DETAILED DESCRIPTION 
       [0015]      FIG. 1  illustrates an exemplary mobile machine  10 . In the disclosed example, machine  10  is a locomotive. However, it is contemplated that machine  10  may embody another type of machine, if desired. For example, machine  10  may embody an on- or off-highway haul truck, a construction machine, or a vocational vehicle. Alternatively, machine  10  could be a stationary machine, such as a genset, a pump, or a drill that requires continuous attention from an operator. Other types of machines may also be possible. 
         [0016]    As a locomotive, machine  10  may include a car body  12  supported at opposing ends by a plurality of trucks  14  (only one truck  14  shown in  FIG. 1 ). Each truck  14  may be configured to engage a track  15  (shown only in  FIG. 2 ) via a plurality of wheels  16 , and to support a frame  18  of car body  12 . One or more braking devices  20  may be associated with one or all wheels  16  of a particular truck  14 , and any number of engines (not shown) may be mounted to frame  18  within car body  12  and drivingly connected to propel wheels  16 . Control over wheel braking and engine fueling (as well as other locomotive controls) may be provided by way of an operator cabin  22  that is supported by frame  18 . 
         [0017]    The interior of an exemplary cabin  22  is shown in  FIG. 2 . As can be seen in this figure, cabin  22  may house a plurality of input devices  24 . Input devices  24  may be used by the operator to control machine  10  and embody any type of device known in the art. For example, input devices  24  may include, among other things, a throttle configured to control fueling of machine  10  and a brake lever configured to control braking of machine  10 . Input devices  24  may be levers, pedals, wheels, knobs, push-pull devices, touch screen displays, etc. 
         [0018]    In the disclosed embodiment, movements and other activities of a machine operator may be tracked from inside of cabin  22 . These movements and/or activities may include the use of input devices  24 . For example, in addition to generating control signals used to control machine operations, one or more of the signals generated by input devices  24  may also be monitored and used by a fatigue monitoring system  26  (“system”—shown in  FIG. 3 ) as a way to determine if the operator is fatigued. In particular, based on a frequency of input device use, system  26  may be able to determine if the operator is alert (e.g., when input is received frequently), slightly fatigued (e.g., when input frequency is reduced), significantly fatigued (e.g., when input is sporadically received), or even asleep (e.g., when no input is received). In some embodiments, a dedicated input device  24  may used for this purpose. That is, a particular input device  24  may be provided for the sole purpose of demonstrating operator alertness. The machine operator must manipulate this dedicated input device  24  with a minimum frequency in order to show a desired level of alertness. 
         [0019]    In addition to using the frequency of input device  24  use as a way to determine a fatigue level of the machine operator, system  26  may use monitoring hardware for this same purpose. For example, system  26  may include a scanner  28  alone or together with one or more light sources  30 . In the disclosed example, scanner  28  is an IR scanner (e.g., a camera) and light sources  30  produce IR light that is only visible to scanner  28  for use by scanner  28  during low-light conditions. Other types of scanners and/or light sources may alternatively be utilized for this purpose, if desired. Scanner  28  and light sources  30  may be located within cabin  22 , for example within a central control panel facing the machine operator. 
         [0020]    As shown in  FIG. 3 , system  26  may include a controller  32  that is in communication with input devices  24 , scanner  28 , and light sources  30 . Controller  32  may be configured to receive signals generated by these components, determine a level of operator fatigue based on the signals, and selectively activate one or more warning devices  34  and/or a penalty device  36 . In the disclosed example, two warning devices  34  are shown, including an audible warning device and a visual warning device. It is contemplated that any number and/or type of warning device  34  may be used and located anywhere on machine  10  (e.g., within cabin  22 ). Penalty device  36  may be a device used by controller  32  to automatically adjust operation of machine  10  when the operator is determined to be fatigued past a threshold amount (e.g., when the operator is unresponsive). In the disclosed example, penalty device  36  is an actuator functional to automatically activate braking device  20  associated with wheels  16 . In some embodiments, this same actuator (or another actuator) may be used to simultaneously (or separately) reduce fueling of the engine of machine  10 . 
         [0021]    Controller  32  may embody a single microprocessor or multiple microprocessors that include a means for controlling operations of machine  10  (e.g., of system  26 ) in response to received signals. Numerous commercially available microprocessors can be configured to perform the functions of controller  32 . It should be appreciated that controller  32  could readily embody a general machine microprocessor capable of controlling numerous machine functions. Controller  32  may include a memory, a secondary storage device, a processor, a timer, and any other components for running an application. Various other circuits may be associated with controller  32  such as power supply circuitry, signal conditioning circuitry, solenoid driver circuitry, and other types of circuitry. 
         [0022]      FIG. 4  illustrates an exemplary method performed by controller  32 .  FIG. 4  will be discussed in detail in the following section to further explain the disclosed concepts. 
       INDUSTRIAL APPLICABILITY 
       [0023]    The fatigue monitoring system of the present disclosure may be applicable to any machine where operator alertness is important. The disclosed fatigue monitoring system may help to keep the operator alert by monitoring an activity level of the operator and also by monitoring characteristics (e.g,, facial characteristics) of the operator. When the fatigue monitoring system determines that the operator is fatigued, a level of the fatigue may be judged and multiple different actions may be taken to alert the operator and/or to override operator control and ensure safe machine operation. Operation of fatigue monitoring system  26  will now be described in detail. 
         [0024]    During operation of machine  10 , system  26  may be used to determine and respond to a fatigue level of the operator. Controller  32  may initiate this process by starting an internal timer and initiating a scan of the operator (Step  400 ). The scan may be initiated by activating scanner  28  and, in some instances, also activating light sources  30 . Based on the images and/or video produced by scanner  28 , controller  32  may then determine if the operator is fatigued (Step  405 ). In particular, controller  32  may compare the scanned image of the operator with one or more images stored in memory. For example, controller  32  may determine an actual head pose from the scanned image, and compare the actual head pose to one or more head poses corresponding to different levels of operator fatigue. In another example, controller  32  may calculate an eye position or eye blink frequency from the scanned image, and compare the eye position and/or blink frequency to threshold positions or frequencies. Other operator characteristics may also or alternatively be captured in the scanned image and compared to information stored in memory corresponding to the different levels of operator fatigue. 
         [0025]    Based on these comparisons, controller  32  may then determine if the fatigue level of the operator corresponds with a particular level that requires action by controller  32 . For example, controller  32  may determine if the operator is fatigued at a level 1, at a level 2, or at a level 3 (Step  410 ). In the disclosed embodiment, level 1 fatigue may correspond with a slightly fatigued operator wherein response time and/or judgment may be somewhat impaired. Level 2 fatigue may correspond with a significantly fatigued operator wherein response time and/or judgment is significantly impaired. Level 3 fatigue may correspond with an unresponsive operator (e.g,, an operator that has fallen asleep). It is contemplated that any number of fatigue levels may be utilized by system  26 , and/or that the levels of fatigue may have different meanings, if desired. 
         [0026]    When controller  32  determines, based on the scan of the operator, that the operator is fatigued at level 1, controller  32  may activate warning device  34  at a level 1 output and also log a corresponding fault condition (Step  415 ). In the disclosed embodiment, the level 1 warning may include activation of only visual warning device  34 , with the intention to alert and refresh the operator. 
         [0027]    When controller  32  determines, based on the scan of the operator, that the operator is fatigued at level 2, controller  32  may activate warning device  34  at a level 2 output and also log a corresponding fault condition (Step  420 ). In the disclosed embodiment, the level 2 warning may include activation of only audible warning device  34  or activation of both visual and audible warning devices  34 , with the intention to alert and refresh the operator by a greater degree, 
         [0028]    When controller  32  determines, based on the scan of the operator, that the operator is fatigued at level 3, controller  32  may activate penalty device  36  and also log a corresponding fault condition (Step  425 ). In some applications, controller  32  may additionally communicate (e.g., via wireless communications) the fault condition to other personal onboard machine  10  and/or to another location (e.g., to the central service facility). In the disclosed embodiment, activation of penalty device  36  may result in a reduction of engine fueling, activation of braking devices  20 , or both, with the intention to override the operator and slow or stop machine  10 . 
         [0029]    Even if the scan of the operator does not show that the operator is fatigued, the operator could still be fatigued and the fatigue simply not detectable via scanning. Accordingly, if the operator is determined at step  405  to not be fatigued, controller  32  may check the activity level of the operator to determine if a first time threshold has elapsed since input from the operator was received via devices  24  (Step  430 ). If the first time threshold has not elapsed, controller  32  may determine if any input from the operator has been received (Step  435 ). When input from the operator is received, the timer may be reset and control may return to step  400 . Control may cycle through steps  405 ,  430 , and  435  until the input is received or until the first time threshold has elapsed. In the disclosed embodiment, the first time threshold may be about 45 seconds, although other time thresholds may also be utilized. 
         [0030]    If the first time threshold elapses without additional input from the operator, controller  32  may activate the level 1 warning and log the corresponding fault condition (Step  415 ). Controller  32  may then continue monitoring operator input to determine if the input is subsequently received (Step  440 ) and also continue providing the warning at level 1. As with step  435  described above, once the input is received, control may pass to step  400  and the timer may be reset, However, if a second time threshold greater than the first time threshold is determined to have elapsed without operator input (Step  445 ), controller  32  may activate warning device(s)  34  at level 2 and log the corresponding fault condition (Step  420 ). 
         [0031]    Again, controller  32  may monitor operator input after activation of warning devices  34  at level 2 to determine if the input is subsequently received (Step  450 ), continue tracking time, and continue activation of warning devices  34 . As with steps  435  and  440  described above, once the input is received, control may pass to step  400  and the timer may be reset. However, if controller  32  determines that a third time threshold greater than the first time threshold has elapsed without operator input (Step  455 ), controller  32  may activate penalty device  36 , log the fault condition, and/or communicate the log to other personnel or to the offboard service facility (Step  425 ). 
         [0032]    In an alternative embodiment, not shown, controller  32  may be configured to simply detect existence of a fatigue event (i.e., without quantifying the event) based on the scanned image of the operator. And based on existence of the event, immediately initiate the level 2 and/or 3 warnings and/or activate penalty device  36 . Other control strategies may also be possible. 
         [0033]    It is contemplated that system  26  may be the combination of a fatigue based system with an existing alerter system, if desired. For example, the existing alerter system may function based solely on the frequency of input received from the operator until after system  26  visually detects some minimum level of fatigue in the operator. In this embodiment, system  26  may affect operation of the conventional alerter system only after the minimum level of fatigue is detected. And when the minimum level of fatigue is detected, system  26  may immediately initiate a maximum level of audio and/or visual warning. This could help to snap the operator out of the normal fatigue cycle observed aboard train locomotives equipped with conventional alter systems. It is also contemplated that the audio and/or visual warning implemented based on fatigue detection could be different than the warning normally implemented by the existing alerter systems (e.g., a different volume, sound, etc.). 
         [0034]    It is further contemplated that the disclosed system  26  could be associated with a train locomotive event recorder (not shown). For example, each time that a warning is issued and/or when penalty device  36  is activated, the corresponding faults could be logged into the event recorder. This information may then be selectively offloaded from the locomotive to a remote monitoring system, if desired. 
         [0035]    Because system  26  may rely on monitored operator activity and also on scanned images of the operator, the system may be more apt to detect and correctly recognize a fatigue event. This improved ability may help to ensure that a response to such events is quickly implemented. 
         [0036]    It will be apparent to those skilled in the art that various modifications and variations can be made to the fatigue monitoring system of the present disclosure without departing from the scope of the disclosure. Other embodiments will be apparent to those skilled in the art from consideration of the specification and practice of the fatigue monitoring system disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope of the disclosure being indicated by the following claims and their equivalents.