Systems and method of datalink auditory communications for air traffic control

A system and method for engaging in two-way communication between a pilot and an air traffic controller by speech and hearing processes wherein spoken messages are compared to a database of categorized messages for matching and selecting a categorized message for transmission between pilot and air traffic control.

BACKGROUND OF THE INVENTION

Pilots operating an aircraft face an array of instrumentation and controls. Instrumentation and controls often times provide a major distraction to the pilot. This distraction can be very serious as the pilot approaches restricted or congested air space, as occurs in the travel lanes approaching an airport. For example, current displays are often cluttered with email-like messages that demand a pilot's, or ground controller's attention. There exists a need for systems and procedures that can reduce or virtually eliminate the pilot's or controller's distraction from flying and monitoring the status of the aircraft so that safety is improved to everyone onboard the aircraft and the operational efficiency of aircraft monitoring is enhanced.

BRIEF SUMMARY OF THE INVENTION

A system and method of communication utilizing auditory and speaking processes between transmission and reception locations that utilize digital data linked communications including but not limited to communication between a pilot and air traffic controller.

DETAILED DESCRIPTION OF THE INVENTION

The communication systems and methods as will be described apply to any two-way communication between a transmitter and recipient. More particularly, the two-way communication systems include enhancements that improve the communication that occurs between aircraft pilots and air traffic control centers controllers. The enhanced system and method employs speech and auditory processes that are digitally sent, received, and processed such that distraction to the pilot or air traffic controller is minimized. The enhanced system and method converts aviation related messages and protocols spoken by a pilot or air traffic controller that is consistent in meaning to the messages used in Controller Pilot Data Linked Communications (CPDLC). The CPDLC messages are fully described in Safety And Performance Requirement Standards for Air Traffic Data Link Services In Continental Airspace (RTCA/DO-290, Apr. 29, 2004, RTCA Inc.), Interoperability Requirements Standard For ATN Baseline 1, Revision A (DO-280A/EUROCAE ED-110A, Nov. 19, 2004, RTCA Inc.) and Guidance Material For ATS Data Link Services In North Atlantic Airspace (Version 12.0, May 19, 2005, ScOACC, UK), incorporated by references as if fully disclosed herein. In the references above are Standards and Recommended Practices (SARPS) for CPDLC based communication.

In a particular embodiment, a pilot's or ground controller's spoken aviation-related messages having phraseologies relating to aircraft and ground control operations are converted to digitized messages. The aircraft and ground operations include CPDLC procedures applicable during busy times as occurs preparing for and during take offs and landings, as well as during the less busy times while the plane is enroute between airports. The digitized messages are matched to at least one standardized message among sets of standardized messages stored in a microprocessor-executed device. For example, the sets of standardized messages include but is not limited to CPDLC up-linked (UM) and down-linked (DM) messages as described in the referenced RTCA/DO-290 and Version 12.0 documents above. The matched standardized message is presented on a display and is either up-linked communicated to the aircraft or down-linked communicated to the air traffic control center. The displayed message is converted to speech and is listened to by the pilot or ground controller.

Uplinked messages sent from controller to pilot is converted from text to speech so that it is audible by the pilot, and spoken messages from the pilot that are destined to be sent to the controller are converted from speech to text. Either uplink messages from the controller or messages intended for downlink transmission from the pilot to controller are presented on an alphanumerical display for confirmation. The messages either uplinked from the controller or messages intended for downlink transmission from the pilot to the controller are selected according to different flight requirements while the aircraft or center is in operation.

The aircraft and aircraft controller communication systems and methods are further described with reference to the figures below.

FIG. 1Ais a schematic of an uplink communication system10A from a controller to the pilot. The system10A includes a headset11B that is worn by a pilot in an aircraft30and a headset11A worn by an air traffic controller in an air traffic control center60. Each headset11A and11B includes at least one speaker14and a microphone16. A controller wearing headset11A in the air traffic control center60speaks into the microphone16. The microphone16and speaker14are connected to a microprocessor display and transceiver20A. The transceiver20A is in communication with a radio transmission tower40that emits an uplink signal40A to be received by the aircraft30. In the aircraft30, a microprocessor display and transceiver20B receives the uplink signal40A and presents an audio equivalent through the speaker14of the pilot headset11B.

FIG. 1Bis a schematic of a downlink communication from the pilot to the controller. As shown, the communication system10B yields a downlink signal40B from the aircraft60. The pilot talks into the speaker16which is sent to the device20B. The device20B transmits and sends out the downlink radio transmission signal40B the tower40which in turn relays the received signal40B to the device20A.

FIG. 2is schematic diagram of microprocessor and transceiver20B of aircraft30used for transmission of downlink signals40B or reception of uplink signals40A from and to the center60. The microphone16and speaker14are in data communication with a Communications Management Unit (CMU)20-1. The CMU20-1is also in signal communication with a Multi-Purpose Control Display Unit (MCDU)20-12and a Very High Frequency Digital Radio (VDR)20-16as described below. The MCDU20-12provides the interface between the pilot and the CMU20-1.

The CMU20-1includes a speech recognition processor20-2in signal communication with the microphone16, a command processor20-6in signal communication with the speech recognition processor20-2, and a voice read-back processor20-8in signal communication with the command processor20-6and the speaker14. The command processor20-6in turn is in signal communication with the voice read-back processor20-8and separately with the MCDU20-12through which the pilot interacts, and separately in signal communication with the VDR20-16. The VDR20-16generates and transmits a downlink radio signal40B or receives and conveys an uplink radio signal40A transmitted by center60. The MCDU may include a Multi-Function Display (MFD).

FIG. 3illustrates components of the microprocessor and transceiver20A of center60used for reception of downlink signal40B or transmission of uplink signal40A from and to the aircraft30. The downlink electro magnetic signal40B is received by the microprocessor and transceiver20A using a Very High Frequency (VHF) ground station50-2. Alternately the device20A may also be a High Frequency (HF), SATCOM, Gatelink, Broadband Satellite, or datalink sub-networks. The informational content of the transmission40B is routed through the VHF50-2and sent to an Airborne Communication Addressing and Reporting System (ACARS) router50-6or an Aeronautical Telecommunications Network (ATN) router50-10. The ACARS router50-6and the ATN router50-10both converge to a network50-14. The network50-14in concert with the ACARS router50-6and the ATN router50-10serves to link other airline operational control centers managing near and far away aircraft. The network's ACARS and ATN routers50-6,10transmit dispatches and other aeronautical related information to other ATC centers that have or will have operational jurisdiction of a given aircraft30during its flight. The network50-14in turn is in signal communication with an interactive display50-16that serves similar auditory and visual interface tasks with the controller of center60as the MCDU20-12does for the pilot of aircraft30. The interactive display50-16in turn is in data communication with the air traffic controller's headset11, wherein, as described below, speech-to-text processes are engaged with the microphone16and text-to-speech processes are engaged with the speakers14.

FIG. 4is a method flowchart for use in either the aircraft30or the air controller60. The general method begins with block60-2where a CPDLC session is established by speech processes. Thereafter, at process block60-4, the ground controller and the pilot conduct datalink communications by speech and auditory processes. Then the method concludes with process block60-6where the CPDLC session is terminated by a speech command.

FIG. 5is an expansion of sub-algorithm block60-4ofFIG. 4. Beginning from block60-2, block60-4includes several sub-algorithms. First, sub-algorithm block60-4A where the pilot or air traffic controller talks using standard aircraft and control phraseologies consistent with controller-to-pilot uplinked (UM) and pilot-to-controller downlinked (DM) CPDLC messages that are classified according to reference numbers that address various stages of flight operations. For example, as described in the RTCA/DO-290 and Version 12 Documents above, reference numbers UM0-5 concern uplinked responses and acknowledgements, UM6-223 concern uplinked vertical clearances, DM0-5 concern downlinked response and acknowledgements, and DM6-10 concern downlinked vertical clearances. Some messages are automatically generated, for example logical acknowledgement (LACK), DM99 CDA, and others. Other UM and DM CPDLC reference number sets concern other flight operations, such as speed requests, route modification requests, reports, negotiations, emergency messages, etc. In the standard UM and DM messages are phraseologies or message elements having key words that are subsequently be used as triggers for matching with CPDLC reference numbers.

As the controller or pilot speaks the message elements of the respective UM or DM referenced CPDLC messages into the microphone16, there is a speech-to-text conversion that occurs at the next block60-4C along with file storage of the spoken message. The speech-to-text conversion may be achieved by microprocessors configured to run executable programs such as NaturallySpeaking 8 available from Nuance, Inc. (Burlington, Mass., USA). For example, if the pilot requests a weather deviation under CPDLC message DM27 up to a specified distance, the pilot speaks “Request weather Deviation Up To 10000 feet”, wherein “weather deviation” in the partial phrase “weather deviation up to” serves to act as triggering keywords for identifying and selecting a DM27 CPDLC message that will be subsequently sent to a ground controller as discussed below. Similarly, a ground controller who has received the DM27 message speaks “Cleared to deviate up to 10000 feet”, wherein the “Cleared to deviate” serves to act as triggering keywords for identifying and selecting a UM82 CPDLC message.

Continuing withFIG. 5, at block60-4E, the message is retrieved from the stored file and is analyzed for key word content. At process60-4G, the spoken message is compared with stored CPDLC standard message sets, for example the uplinked messages (UM) or downlinked messages (DM) depending if the user is a pilot in aircraft30or controller in center60. The UM and the DM are structured in a defined format with a column designated for intent or use of the UM or DM message element statements. A given message element statement for a given codified UM or DM is contained within bounded brackets that designates words that may be spoken by the pilot or controller that are consistent with the intent or use of the codified UM or DM communications. The bracketed message is interpreted by software utilizing Packed Encoding Rules (PER) under Abstract Syntax Notation-1 (ASN-1) as described in the SARPS of the referenced in the RTCA/DO-290 and Version 12.0 documents cited above. The meaning of a speaking pilot or speaking ground controller phraseologies are ranked according to contextual meaning under utilizing the PER standards. Contextual management of the stored CPDLC contextual-ranked message sets is established by retrieving from a database and comparing keyword triggers of the pilot's or controller's spoken message, at block60-4H to determine if there is a CPDLC standard message that matches the spoken message. If the result of block60-4H is no, then the process returns back to block60-4A where the user presents another message. If the result is yes, that there is a match, then at process block60-4J the CPDLC matched message is displayed on the MCDU20-12. Once the message is displayed, another decision diamond is reached at decision diamond60-4K, which determines if the system is to read-back the CPDLC displayed message. If no, then at process block60-4R, the CPDLC match message that is displayed is sent out as a radio transmission uplink signal40A or downlink signal40B. If yes, the matched CPDLC message that is displayed is read back at process block60-4L. The read back process utilizes text-to-speech conversion that may be achieved microprocessors configured to run executable programs such as Vocalizer 4.0 available from Nuance, Inc. (Burlington, Mass., USA). Thereafter, another decision diamond60-4P is reached in the algorithm in which the query is presented whether or not the CPDLC message is acceptable to the user. If it is acceptable, the process at block60-4R is performed and the matched CPDLC message is outputted as a radio transmission signal40A or40B. If the message is not acceptable, then the process returns to block60-4A where the user speaks another message. An alternate embodiment of the matched algorithm allows for secondary, tertiary and other ranking CPLDC messages with the spoken message to be offered for the pilot's or the air traffic controller's consideration as an appropriate message using software algorithms that rank the message elements by PER and ASN-1

FIG. 6is an alternate process70for establishing, conducting, and terminating a CPDLC session. The process begins at block71with a message that is spoken. The spoken message is recorded, digitized and stored in memory. Retrieval of the message occurs at block72and the message is presented as displayed text on the MCDU20-12, see block74. A decision diamond76then follows with a query whether or not to read back the presented message. If the answer is “no”, then at block82the message is sent out as either uplink signal or downlink signal. If the query is “yes” to read back the message, then at process block78, the message is read back which is followed again by a query if the read-back message is acceptable, see decision diamond80. If the message is acceptable, then the message is sent at process block82as either uplink signal or downlink signal or alternatively, if the answer is “no”, a message “No CPDLC message” is presented to the operator. Upon hearing “No CPDLC message”, the pilot or ground controller speaks a new message at process block71and the process70repeats. The methods ofFIGS. 5 and 6may also be applied to the voice-activation of screen regions or image icons located on the MCDU20-12display20alocated in the aircraft30, or other display located in the center60, for example push button icons, that result in the sending of signals40A and/or40B.

FIGS. 7-9depict an example of the application of the system and methods of the voice triggered CPDLC messages occurring between the pilot in the aircraft30and controller in the air traffic center60. Messages from both pilot and controller are shown depicted in a MCDU90and an interactive display92of the respective aircraft30and aircraft control center60. The scenario series as depicted include a pilot initiated request conveyed in a CPDLC downlinked message contained within a signal98, a controller response conveyed with a CPDLC uplink message contained within an uplink signal100, and a confirmation from the pilot in a follow-up downlink message contained within downlink signal102.

FIG. 7schematically depicts a downlink communication request contained within signal98from the pilot using auditory and speaking processes conducted during a CPDLC session to the air traffic controller. In this scenario, the pilot is requesting a flight deviation for a specified nautical mile (NM) range. The downlink signal98from the aircraft30contains a matched CPDLC statement DM27 as a consequence of the pilot speaking “Request weather deviation up to 30 nautical miles left of route” into the pilot's microphone16B that is speech-to-text converted and presented on the MCDU90as pilot screen view106. The alphanumeric text of the pilot's spoken statement in screen view106is presented as REQUEST WEATHER DEVIATION UP TO 30 NM LEFT OF ROUTE and is available for pilot viewing or read-back to the pilot on the pilot's speaker14B. Once the message is reviewed (e.g., aural read-back or display) then the pilot says “Send”, which sends the downlink message to the ATC Controller.

In the aircraft30, the downlink signal98is processed and presented on the interactive display92as AIRCRAFT HON1234 REQUEST WEATHER DEVIATION UP TO 30 NM LEFT OF ROUTE. In the control center60, the controller is then presented with a text-to-speech conversion of the displayed controller message, see call-out window. In one embodiment abbreviations are vocally converted to the nonabbreviated version (e.g. NM=nautical miles). Beneath the received message is a prompt “Response?” to remind the controller to prepare a voice-triggered CPDLC response message to the pilot's query.

FIG. 8schematically depicts an uplink CPDLC communication response conveyed in signal100from the controller to pilot's query ofFIG. 7. In this case, the controller speaks “Aircraft HON1234 is cleared to deviate up to 30 nautical miles left of route” into controller's microphone16that is subsequently speech-to-text converted. Information from the controller response is sent to the aircraft30in CPDLC standard message UM82 in the uplink signal100. The controller's spoken message is processed through a voice recognition processor and is presented adjacent to the “Response?” as AIRCRAFT HON1234 IS CLEARED TO DEVIATE UP TO 30 NM LEFT OF ROUTE. Status is also indicated—unconfirmed.

In aircraft30, uplink signal100is received, processed, and presented on the MCDU90as alphanumeric text AIRCRAFT HON1234 IS CLEARED TO DEVIATE UP TO 30 NM LEFT OF ROUTE. In the aircraft30, the pilot then hears the text-to-speech conversion, see the call-out window above the MCDU90. The pilot then readies a confirmation message to controller's response to pilot's query.

FIG. 9schematically depicts a downlink CPDLC confirmation response conveyed in signal102from the pilot to the controller's response ofFIG. 8. In response, pilot responds “Wilco”, meaning will comply, into pilot's microphone16B and the speech to text message is presented on the MCDU90as WILCO. The information content for “Wilco” is conveyed to controller at center60via CPDLC downlink message DM0 contained within downlink signal102. Pilot confirmation is displayed as status: CONFIRMED on the interactive display92. The pilot confirmation is converted to speech, see call-out window above the interactive display92(“HON1234 will comply with clearance to deviate.”).

The foregoing examples covered inFIGS. 7-9may also apply to ATC Logon and establishment of CPDLC connections. Moreover, some messages that require short responses like “Notify”, “Logon”, “Perform Logon”, “Roger”, “Affirm”, “Accept”, “Reject” before a message is sent may be reviewed by the speaking pilot or speaking ground controller and have the option to say “send”, “cancel”, or “respond later”.

While the particular embodiments of the invention have been illustrated and described, as noted above, many changes can be made without departing from the spirit

and scope of the invention. For example, voice recognition with or without alphanumeric or biologically based passwords or identifiers may be employed as a security measure to identify the speaking pilot or ground controller instead of speech recognition which doesn't necessarily identify the speaker. Moreover, the enhanced system and method for air traffic control permits a single aircraft having multiple CPDLC logons and sessions with more than one ATC. Similarly, a single ATC controller may have multiple CPDLC sessions with more than one aircraft. This multi-session CPDLC process may be accomplished by the respective speaking pilot or air traffic controller voicing the designation or ID numbers of the appropriate ATC or aircraft entity. Accordingly, the scope of the invention is not limited by the disclosure of the preferred embodiment. Instead, the invention should be determined entirely by reference to the claims that follow.