Patent Description:
Contemporary aircraft travel flight paths that generally includes a climb, a cruise, and a descent. Pilots along with flight management systems (FMS) implement the flight plan. The FMS may generate the flight plan by taking into account parameters specific to the aircraft and to the flight conditions such as the payload, the weight of the aircraft, the quantity of fuel onboard, temperature, wind, altitude, etc., and of the time constraints imposed by the air traffic control. The flight plan may describe all the waypoints or positions through which the aircraft is to pass, with the altitude and corresponding speed at each waypoint.

<CIT> discloses a method of controlling an aircraft in the event of pilot incapacity, The method includes detecting, using at least one sensor, the occurrence of an event relating to a pilot's capacity to control the aircraft, determining whether the event justifies a controlled takeover of the aircraft from the pilot, asserting a controlled takeover of the aircraft from the pilot if it has been determined that a controlled takeover is justified, and executing a controlled landing of the aircraft without assistance of the pilot.

<CIT> discloses a system and method for intelligently managing the avionics display, information, and controls to more evenly distribute pilot task loads and/or automatically configure/reconfigure displays during flights.

Claim <NUM> defines a method of adjusting a planned pilot workload before the planned flight. In the following, apparatus and/or methods referred to as embodiments that nevertheless do not fall within the scope of the claims should be understood as examples useful for understanding the invention.

Aspects of the present disclosure are directed to a method of adjusting a pilot workload during a flight, not being part of the present invention.

As used herein, workload means the tasks a pilot must perform during various stages of the flight. During operation of an aircraft, portions of a flight plan such as takeoff, landing or even flying over certain geographic areas can require significant pilot attention and workload and in some cases can increase a pilot's stress level. Other portions of a flight plan, such as cruise during long flights, may need minimal pilot attention. It can be beneficial to track a pilot's workload or stress level during flight and adjust the pilot's workload when the pilot's stress level or workload exceeds a predetermined threshold.

The present invention is directed to a method of adjusting a pre-planned workload before a fight. Since flights often take place over the same flight path, such as from city-to-city, flight data can be collected that is representative of a typical flight. During any given flight, data representing a pilot's workload or stress level can be collected and correlated with portions of a flight plan such as takeoff, landing or flying over certain geographic areas. Statistical analysis of the data can determine when a pilot has experienced high workload volumes or high stress levels. The data can be saved and analyzed to understand when a pilot will likely experience increases in workload for a similar future planned flight. Prior to a future flight, it can be beneficial to understand when a pilot will likely experience higher workloads during the flight. In this scenario, it may be possible to adjust the expected pilot work load during heavy workload periods by shifting tasks during the heavy workload period to light workload periods. Moreover, understanding workloads during a flight can help airlines adjust crew rosters to adjust for workload conditions.

For purposes of illustration, the present disclosure will be described in the context of a flight management system in an aircraft environment. It will be understood, however, that the disclosure is not so limited and may have general applicability in non-aircraft applications, such as other mobile applications.

As used herein, "a set" can include any number of the respectively described elements, including only one element. All directional references (e.g., radial, axial, proximal, distal, upper, lower, upward, downward, left, right, lateral, front, back, top, bottom, above, below, vertical, horizontal, clockwise, counterclockwise, upstream, downstream, forward, aft, etc.) are only used for identification purposes to aid the reader's understanding of the present disclosure, and do not create limitations, particularly as to the position, orientation, or use of the disclosure. Connection references (e.g., attached, coupled, connected, and joined) are to be construed broadly and can include intermediate members between a collection of elements and relative movement between elements unless otherwise indicated. As such, connection references do not necessarily infer that two elements are directly connected and in fixed relation to one another. The exemplary drawings are for purposes of illustration only and the dimensions, positions, order, and relative sizes reflected in the drawings attached hereto can vary.

As used herein, a "controller" can include at least one processor and memory. Non-limiting examples of the memory can include Random Access Memory (RAM), Read-Only Memory (ROM), flash memory, or one or more different types of portable electronic memory, such as discs, DVDs, CD-ROMs, etc., or any suitable combination of these types of memory. The processor can be configured to run any suitable programs or executable instructions designed to carry out various methods, functionality, processing tasks, calculations, or the like, to enable or achieve the technical operations or operations described herein. The program can include a computer program product that can include machine-readable media for carrying or having machine-executable instructions or data structures stored thereon. Such machine-readable media can be any available media, which can be accessed by a general purpose or special purpose computer or other machine with a processor. Generally, such a computer program can include routines, programs, objects, components, data structures, algorithms, etc., that have the technical effect of performing particular tasks or implement particular abstract data types.

<FIG> schematically illustrates an aircraft <NUM> according to various aspects described herein. One or more propulsion engines <NUM> can be coupled to a fuselage <NUM>, a cockpit <NUM> can be positioned in the fuselage <NUM>, and wing assemblies <NUM> can extend outward from the fuselage <NUM>. A plurality of aircraft systems <NUM> that enable proper operation of the aircraft <NUM> can be included as well as a flight control computer <NUM> (or "computer" <NUM>). While a commercial aircraft has been illustrated, it is contemplated that aspects of the disclosure can be used in any type of legacy aircraft, for example, without limitation, fixed-wing, rotating-wing, rocket, or personal aircraft.

The plurality of aircraft systems <NUM> can reside within the cockpit <NUM>, within an electronics and equipment bay <NUM>, or in other locations throughout the aircraft <NUM> including that they can be associated with the engines <NUM>. Such aircraft systems <NUM> can include but are not limited to: an electrical system, an oxygen system, hydraulics and/or pneumatics system, a fuel system, a propulsion system, navigation systems, flight controls, audio/video systems, an Integrated Vehicle Health Management (IVHM) system, and systems associated with the mechanical structure of the aircraft <NUM>. A variety of aircraft systems <NUM> have been illustrated for exemplary purposes and it will be understood that they are only a few of the systems that can be included in the aircraft <NUM>.

A data network <NUM> over which the plurality of aircraft systems <NUM> can communicate with each other and provide information to a crew of the aircraft <NUM> can be included. For example, the aircraft systems <NUM> can output various information to a flight deck <NUM> located in a cockpit <NUM> of the aircraft <NUM>.

A communication interface <NUM> can be located within the aircraft <NUM> and operably coupled to at least some of the plurality of aircraft systems <NUM>. The communication interface <NUM> has been illustrated as being included in the cockpit <NUM>. It is contemplated that the communication interface <NUM> can be located in other locations within the aircraft <NUM> including within the electronics and equipment bay <NUM>. Although only one communication interface <NUM> has been illustrated, it is contemplated that the aircraft <NUM> can have multiple communication interfaces. The communication interface <NUM> can be utilized for communication with other aircraft or a ground station <NUM>, such as by radio contact in a non-limiting example. In addition, the communication interface <NUM> can transmit or receive data, including audio or visual data as appropriate.

The ground station <NUM> can be in communication with the data network <NUM> or communication interface <NUM>. The ground station <NUM> can have a processor and software and capabilities for uploading or downloading software or data to the aircraft. The ground station <NUM> might also capability to store, analyze, and manipulate flight data with data analysis or other statistical software. It should be recognized that the aircraft <NUM> might also have a computing system capable of storing, analyzing, and manipulating flight data with data analysis or other statistical software.

<FIG> illustrates a portion of the cockpit <NUM> of the aircraft <NUM> and an exemplary flight deck <NUM> having various instruments <NUM> and flight displays <NUM>. A first pilot (herein, a "pilot") can be present in a seat <NUM> at the left side of the cockpit <NUM> and a second pilot (herein, a "co-pilot") can be present at the right side of the cockpit <NUM> in a seat <NUM> and the flight deck <NUM> can be located in front of the pilot and co-pilot and can provide the flight crew with information to aid in operating the aircraft <NUM>. The flight displays <NUM> can include either primary flight displays or multi-function displays and can display a wide range of aircraft, flight, navigation, and other information used in the operation and control of the aircraft <NUM>. Further, both the various instruments <NUM> and flight displays <NUM> of the flight deck <NUM> can provide one or more visual indicia indicative of a corresponding health condition of one or more of the aircraft systems <NUM>.

The instruments <NUM> and flight displays <NUM> can be laid out in any manner including having fewer or more instruments or displays. Further, the flight displays <NUM> need not be coplanar and need not be the same size. A touch screen display or touch screen surface can be included in the flight display <NUM> and may be used by one or more flight crew members, including the pilot and co-pilot, to interact with the systems of the aircraft <NUM>. Such touch screen surface can take any suitable form including that of a liquid crystal display (LCD) and can use various physical or electrical attributes to sense inputs from the flight crew. It is contemplated that the flight display <NUM> can be dynamic and that one or more cursor control devices <NUM> and/or one or more multifunction keyboards <NUM> can be included in the cockpit <NUM> and can be used by one or more flight crew members to interact with the systems of the aircraft <NUM>. In this manner, the flight deck <NUM> can be considered a user interface for the aircraft systems <NUM> and the aircraft <NUM>.

The flight control computer <NUM> can be operably coupled to components of the aircraft <NUM> including the aircraft systems <NUM>, instruments <NUM>, flight displays <NUM>, touch screen surfaces, cursor control devices <NUM>, keyboards <NUM>, etc. The flight control computer <NUM> can receive inputs from any number of aircraft systems <NUM> or software programs responsible for managing the acquisition and storage of data. The flight control computer <NUM> can also be in the form of a controller, and can be connected with other controllers of the aircraft <NUM>. The flight control computer <NUM> can include memory <NUM> and processing units <NUM>, which can be running any suitable programs to implement a graphical user interface (GUI) and operating system. The flight control computer <NUM> can include or be associated with, any suitable number of individual microprocessors, power supplies, storage devices, interface cards, auto flight systems, flight management computers, and other standard components. The flight control computer <NUM> can include or cooperate with any number of software programs (e.g., flight management programs) or instructions designed to carry out the various methods, process tasks, calculations, and control/display functions necessary for operation of the aircraft <NUM>.

The communication interface <NUM> can be communicably coupled to the flight control computer <NUM> or other processors of the aircraft <NUM> as well as any number of the plurality of aircraft systems <NUM> to transfer information on and off the aircraft <NUM>. The communication interface <NUM> can include any desired communication mechanism capable of wirelessly linking with other systems and devices, such as by radio contact in a non-limiting example. For example, one of the aircraft systems <NUM> can be in the form of a distress tracker <NUM> configured to transmit a state of aircraft distress (for example, "normal," "abnormal," or "distressed").

A pilot tracking system or tracking system <NUM> is illustrated as being in communication with the flight control computer <NUM>. It will be understood that the tracking system <NUM> can be hard wired to the flight control computer <NUM> or can communicate in any suitable manner with the flight control computer <NUM> including via wireless communication. Alternatively, the tracking system <NUM> can be included as a module within the flight control computer <NUM>.

The tracking system <NUM> can include at least one imaging module <NUM> and at least one audio module <NUM>. The imaging module <NUM> can include an image sensor <NUM> configured to sense visual information about a pilot or co-pilot, such as rapid eye movement, eyes moving one direction or another, being open or closed, a direction of gaze, or a facial state such as eyebrows raised or lowered, by way of non-limiting examples and provide an output signal based thereon. An eye movement or body movement parameter can be stored by the tracking system <NUM> The imaging module <NUM> or the flight control computer <NUM> can also be in signal communication with any of the flight displays <NUM>, such as to display a visual indication based on the sensed visual information from the imaging module <NUM>.

The audio module <NUM> can include an audio sensor <NUM> configured to sense audio information about a pilot or co-pilot, such as a language being spoken in the cockpit <NUM>, a voice volume, slurred or altered speech, a speech pattern, or sounds that can occur due to pilot or co-pilot interaction with an aircraft system <NUM> such as striking the flight deck <NUM> or typing on the multifunction keyboard <NUM> and provide an output signal based thereon. The audio module <NUM> can also provide audio feedback or sound to the pilot or co-pilot, such as through speakers mounted within the cockpit or through headphones worn by the pilot or co-pilot. Further, the audio module <NUM> can be in signal communication with the imaging module <NUM>. For example, the imaging module <NUM> can provide an indication for transmission through the audio module <NUM>, such as spoken commands in a low-visibility environment within the cockpit <NUM>. The audio module <NUM> can also provide a signal for transmission via the imaging module <NUM>, such as a flashing light display or text-based indicator to be read by a pilot or co-pilot.

At least one biometric sensor <NUM> configured to sense a biometric parameter of the pilot or co-pilot can be included in the tracking system <NUM>. For example, a biometric sensor <NUM> positioned on the first seat <NUM> can be configured to sense or detect a heart rate, breathing rate, perspiration rate, or bodily motion of the pilot when the pilot is sitting in the first seat <NUM>. Alternatively, the biometric sensor <NUM> can be positioned on a wearable device such as a wrist strap or headband. In still another example, the biometric sensor <NUM> can be in the form of an optical sensor, e.g. a camera that monitors the pilot or co-pilot.

In addition, a seat tracking module <NUM> can control a seat distance <NUM> between the first seat <NUM> and the flight deck <NUM>. While not shown, the second seat <NUM> can also include such a seat tracking module <NUM>. Further, a seat belt sensor <NUM> can sense the position of a seat belt <NUM> on the first seat <NUM> or the second seat <NUM>, such as the seat belt <NUM> being buckled or unbuckled.

A haptic feedback generator <NUM> can be coupled or integrated with either or both of the first and second seats <NUM>, <NUM>. The haptic feedback generator <NUM> can be configured to vibrate, such as a steady or varying vibration pattern, to provide feedback to the pilot or co-pilot. In a non-limiting example wherein the aircraft <NUM> is not level during flight in low-visibility conditions, the haptic feedback generator <NUM> can vibrate on a right-hand portion or left-hand portion of the seat <NUM>, <NUM> to indicate to a pilot or co-pilot which direction to bank the aircraft <NUM> for correct orientation when flying.

A timer <NUM> can also be included in the tracking system <NUM> and is illustrated as being coupled to the flight deck <NUM>. The timer <NUM> can be positioned anywhere within or outside of the cockpit <NUM>. The timer <NUM> can be configured to track an elapsed time of an event or to provide an alarm or other indication at a predetermined time. Non-limiting examples where the timer <NUM> can be utilized include tracking an elapsed time of flight, an elapsed time of a pilot interaction with an aircraft system <NUM> (such as updating flight records via the multifunction keyboard <NUM>), tracking an elapsed sleep time, indicating a time for changing a flight direction, or indicating a wake time.

An additional controller <NUM> having a processor <NUM> and a memory <NUM> can also be included in the tracking system <NUM>. The controller <NUM> is illustrated as being coupled to the flight deck <NUM> and in signal communication with any or all of the flight control computer <NUM>, instruments <NUM>, flight displays <NUM>, memory <NUM>, processing unit <NUM>, imaging module <NUM>, audio module <NUM>, biometric sensor <NUM>, seat tracking module <NUM>, seat belt sensor <NUM>, haptic feedback generator <NUM>, or timer <NUM>. Dashed lines have been used to illustrate a portion of the signal connection between the above-described components. The lack of a dashed line for signal connections is done for clarity of the figures, and it will be understood that components not connected by a dashed line can nonetheless be in signal communication.

<FIG> schematically illustrates components of the tracking system <NUM> in exemplary communicative connection, where a separate controller <NUM> is shown in signal connection with the various modules and sensors described above. Alternatively, it will be understood that the flight control computer <NUM> can be utilized or that each module or any combination of modules can include their own controller, processor, or memory. Arrows have been included to indicate an exemplary signal or control direction, and are provided for clarity in discussion. It will be understood that any signal communication or control between connected components in <FIG> can be transmitted in either direction, and an arrow's illustrated direction does not imply a one-way signal or control direction.

A door lock module <NUM> with a door lock sensor <NUM> can further be included in the tracking system <NUM>. For example, a cockpit door <NUM> can include the door lock module <NUM> configured to sense whether the door <NUM> is locked or unlock. The module <NUM> can also automatically lock or unlock the door <NUM> based on a control signal within the tracking system <NUM>.

The controller <NUM> can be operably connected to and receive input from any or all of the flight control computer <NUM>, image sensor <NUM>, audio sensor <NUM>, biometric sensor <NUM>, seat tracking module <NUM>, seat belt sensor <NUM>, haptic feedback generator <NUM>, timer <NUM>, or door lock module <NUM>. Any of the inputs received by the controller <NUM> can be stored in the memory <NUM>. For example, the memory <NUM> can store a history of audio input or recordings from data gathered within the cockpit <NUM>, or an elapsed time since a pilot last interacted with a flight display <NUM> (<FIG>).

The processor <NUM> of the controller <NUM> can send a signal or control command to any or all of the imaging module <NUM>, audio module <NUM>, seat tracking module <NUM>, haptic feedback generator <NUM>, or door lock module <NUM>. In another non-limiting example, the processor <NUM> can send a signal to the imaging module <NUM>, such as a visual message to be read by the pilot or co-pilot on the flight display <NUM>, or a command to enable or disable the image sensor <NUM>.

It is further contemplated that the controller <NUM> of the tracking system <NUM> can issue a signal or command to another aircraft system, such as the communication interface <NUM> via the flight control computer <NUM>. In such a case, the controller <NUM> can communicate with an external aircraft or ground station (not shown). The controller <NUM> can also be communicatively coupled with any other aircraft system <NUM> as desired.

Referring now to <FIG>, a method <NUM> of adjusting pilot workflow during a flight is illustrated. The method <NUM> begins at <NUM> with tracking inputs by a pilot or tracking image sensor <NUM> data or a biometric parameters of the pilot to determine a stress level of the pilot during flight of the aircraft <NUM> by using at least one sensor within the tracking system <NUM>. The tracking of inputs, sensor data or biometric parameters can occur over distinct periods of time such as over an entire flight, or portions of a flight such as an elapsed time over a geographic location, an elapsed time a pilot has been on-board, or an elapsed time of a flight phase such as flight take off, flight landing or cruising.

Tracking inputs by the pilot can be an indication of the pilot's workload. For example, tracking pilot input can be accomplished via the image sensor <NUM>, such as visually detecting the pilot's hand interacting with the keyboard <NUM> or flight display <NUM>. In an example where the flight display <NUM> includes a touchscreen, the controller <NUM> can detect that the pilot has touched or interacted with the touchscreen flight display <NUM>. Tracking pilot input can also be accomplished via the flight control computer <NUM> which is communicatively coupled to the controller <NUM>, including tracking that the pilot has interacted with or entered input to any cockpit system e.g. the keyboard <NUM> or the cursor control device <NUM>.

Tracking a pilot's stress level can be accomplished by tracking a pilot's eye movements or body movements from the image sensor <NUM> or by tracking various biometric parameters of the pilot. For example, the image sensor <NUM> can track both the eye movements and body movements of the pilot during all or portions of the flight. In addition, biometric parameters of the pilot such as heart rate, breathing rate, skin temperature, and the like, can be indicative of the pilot's stress level. It should be recognized that this data can be used to make real-time decisions about pilot workload or stress level during a flight or the data can be saved to a database and processed after the flight.

At <NUM>, the controller <NUM> can determine the workload or stress level of the pilot based on the tracked pilot input, image sensor <NUM> data, or biometric parameter(s) from <NUM>. For example, the timer <NUM> in conjunction with the flight control computer <NUM> can indicate to the controller <NUM> that the pilot has input multiple inputs into various avionics systems over a short period of time and is backed up on flight control computer <NUM> requests. The controller <NUM> can then determine the rate at which the pilot is addressing flight control computer <NUM> requests and determine the workload level of the pilot.

The controller <NUM> can also or alternatively determine, at <NUM>, a stress level of the pilot based image sensor <NUM> data. For example, if the pilot's eyes are rapidly moving from screen to screen the controller <NUM> can determine the pilot stress level by correlating the movements with other real-time parameters, against pre-existing data, or above a predetermined threshold level.

Similarly, the controller <NUM> can also or alternatively determine, at <NUM>, a stress level of the pilot based on the biometric parameter. For example, the biometric sensor <NUM> can determine a pilot's rate of breathing, heart rate, pulse rate, and the like. The controller <NUM> can determine the pilot stress level by correlating any of the biometrics parameters with other real-time parameters such as data from the image sensor <NUM>, against pre-existing data, or above a predetermined threshold level.

At <NUM>, the controller <NUM> can adjust the pilot workload during flight when one of the pilot workload or pilot stress level exceeds a predetermined threshold. For example, if the controller <NUM> determines that the pilot's stress level is above a predetermined threshold value based on sensor readings from one of the image sensor <NUM> or biometric sensor <NUM> then the controller <NUM> can be programmed to shift certain tasks such as non-critical or non-essential pilot tasks to the co-pilot or other crew member. In addition or alternatively, the controller <NUM> can be programmed to shift the non-critical or non-essential pilot tasks to a future time frame when the pilot has less tasks in his workload.

Turning to <FIG>, a method <NUM> of adjusting a planned pilot workload before the planned flight is shown. The method includes, at <NUM>, tracking inputs by a pilot, or tracking image sensor <NUM> data, or a biometric parameter of the pilot during flight of the aircraft <NUM> during one or more aircraft flights. Since flights often take place over the same flight path, such as from city-to-city, flight data can be collected that is representative of a typical flight between the two cities. At this step, it is contemplated that data can be tracked and collected from multiple flights over various time frames and preferably from flights that fly the same or similar flight paths. The data can be tracked, collected, and analyzed to understand time frames during a flight that pilot's experience heavy workloads and high stress. The data processing can be done on or off aircraft <NUM>, where the flight control computer <NUM> or other processors on the aircraft <NUM> can adjust the expected pilot workload before the planned flight.

At <NUM>, data from the image sensor <NUM>, biometric sensor <NUM> (i.e. biometric parameters), and pilot inputs responsive to the flight control computer <NUM> can be correlated with one or more time periods of the flight. In other words, at this step <NUM>, for any given flight, data representing a pilot's workload or stress level can be collected and correlated over distinct periods of time such as over an entire flight, or portions of a flight such as an elapsed time over a geographic location, an elapsed time a pilot has been on-board, or an elapsed time of a flight phase such as flight take off, flight landing or cruising. In this example, the timer <NUM> can track the time of flight which can be correlated with the collected pilot data across the all or a portion of the flight. Data from each recorded flight can be stored onboard or can be transmitted offsite to a ground station <NUM> for analysis and processing.

At <NUM>, analysis of the data, either on-aircraft by an on-board processor or offsite by the ground station, can identify which of the time periods the pilot's workload was above a predetermined threshold value indicative of a pilot experiencing high workload volumes or high stress levels. In one example, data from multiple independent flights with similar flight paths can be analyzed to understand specific time frames the pilots of each of flights experienced high workload volumes or stress levels. Using this data, the controller <NUM> or other processor can predict when a pilot on a planned flight will likely experience increases in workload or higher stress levels.

At <NUM>, prior to a planned flight, the controller <NUM> or other processer that is operatively connected to the flight control computer can adjust the expected pilot work load during the planned flight. In this scenario, the controller <NUM> or other processer can predict heavy workload periods in the planned flight and can adjust the pilot workload by shifting tasks from the predicted heavy workload period to periods where the pilot workload if predicted to be light. Moreover, the controller <NUM> or other process can predict when work load will be heavy for the entire crew and can predict when additional or less crew may be required for certain flights or certain conditions.

To the extent not already described, the different features and structures of the various embodiments can be used in combination, or in substitution with each other as desired. That one feature is not illustrated in all of the embodiments is not meant to be construed that it cannot be so illustrated, but is done for brevity of description. Thus, the various features of the different embodiments can be mixed and matched as desired to form new embodiments, whether or not the new embodiments are expressly described. All combinations or permutations of features described herein are covered by this disclosure.

Claim 1:
A method (<NUM>) of adjusting a planned pilot workload before a planned flight, the method comprising:
tracking (<NUM>) over a time period, using at least one sensor (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>), inputs by a pilot indicative of a pilot workload, or image sensor data or a biometric parameter indicative of a pilot stress level, during a plurality of flights of an aircraft prior to the planned flight, the plurality of flights having the same or similar flight paths;
correlating (<NUM>), using a controller (<NUM>) that is operatively connected to the at least one sensor (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>), the inputs by a pilot indicative of a pilot workload, or the image sensor data or the biometric parameter indicative of a pilot stress level, with one or more time periods for analysis;
identifying, based on the analysis, using a processor (<NUM>) that is operatively connected to a controller (<NUM>), which of the one or more time periods comprise pilot workload or stress level above a threshold value indicative of a pilot experiencing high workload volumes or high stress levels; and
adjusting (<NUM>), using a controller (<NUM>) that is operatively connected to the flight control computer (<NUM>), the pilot workload during the time periods that comprise a pilot workload or stress level above the threshold value of pre-planned pilot workload of the planned flight.