Patent Description:
Driving a vehicle, whether a road-based vehicle such as a truck or an aerial vehicle such as an aircraft, requires the attention of the vehicle driver (or vehicle operator). A vehicle driver who has too many tasks to undertake while driving a vehicle (e.g., high driver workload), particularly at critical times, can become distracted and a vehicle mishap may occur as a result. Communication between a vehicle driver and other entities may be necessary at times but can also be distracting at times. Multiple communication modalities, such as voice or text, may be available for the vehicle driver. Use of one modality may be more effective for communication but at the same time may increase driver workload and be more distracting for the vehicle driver. For example, a text message may be a more effective modality for communicating at a given time for a truck driver but may also be a more distracting form of communication for the truck driver.

In the case of air travel, there is a desire to decrease flight crew workload to reduce the number of potential flight related mishaps. Additionally, many tasks presently performed by multiple flight crew members, if simplified, could help enable single pilot operation. Communication between flight crew and other entities, if simplified, could help with a move toward single pilot operation. Communication between flight crew and other entities may involve multiple communication modalities. When communicating with different entities, the flight crew often needs to identify one modality out of multiple modality mechanisms available to use for communication, often at critical times. Having to determine the communication modality to use can increase flight crew workload.

Hence, it is desirable to provide systems and methods for automatically choosing communication modalities. Furthermore, other desirable features and characteristics of the present invention will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings and the foregoing technical field and background.

<CIT>, discloses an aircraft instrumentation system for voice transcription of radio communications and identification of commonly communicated data, that has a controller for controlling display device to display a transcript of a voice communication. <CIT> B <NUM> discloses a method for automatically selecting a mode for a detected communication. <CIT> discloses a method for providing a unified communication event management of communication events to receive, e.g. instant messages. <CIT> discloses a computer-based method for providing text-based direction confirmation of vehicle. <CIT> discloses a method for converting a message of a predetermined protocol to a selected protocol.

The invention is disclosed in the independent claims <NUM> and <NUM>. Additional aspects of the invention are disclosed in the dependent claims.

The subject matter described herein discloses apparatus, systems, techniques and articles for reducing the workload for communication between a vehicle operator (e.g., a pilot, truck driver, etc.) and a central system (e.g., air traffic controller, truck logistic center, etc.). The apparatus, systems, techniques and articles disclosed herein can provide a fully integrated voice and data communication management system. The apparatus, systems, techniques and articles provided herein can contribute to reducing workload during communications between a vehicle operator and a central system. The apparatus, systems, techniques and articles provided herein can contribute to reducing workload during communications by: automatically selecting the best communication modality for communications between communication participants (e.g., vehicle operator and central system); reducing the analyzing time of messages by communication participants through a unified representation of incoming voice and data messages in a consistent way; and enabling a more systematic automation of actions driven by messages (whether they are received by data or voice).

The apparatus, systems, techniques and articles provided herein can contribute to increasing safety by avoiding confusion in communication. The apparatus, systems, techniques and articles provided herein can contribute to decreasing operation costs: by allowing the vehicle owner (and vehicle operating center such as an airline operating center) to optimize communication costs by selecting the most cost efficient mode of communication at a given time. The apparatus, systems, techniques and articles provided herein can contribute to decreasing vehicle costs by allowing use of fully digital solutions for both data and voice.

<FIG> is a data flow diagram depicting an example process in a unified communication system <NUM> for use by a message originator <NUM> for sending messages to one or more message recipients <NUM>. The message originator <NUM> may be a vehicle operator such as a pilot (or other flight crew member), a truck driver, or some other type of vehicle operator who may need to communicate with a plurality of different control centers (e.g., ATC) during a mission. Different modalities may be used to communicate with the different control centers. The unified communication system <NUM> is configured to automatically select a communication modality for use when communicating with a particular control center freeing the vehicle operator to focus his/her attention on matters other than choosing the correct modality for the particular control center.

Alternatively, the message originator may be a control center that needs to communicate with a plurality of different vehicles that use different communication modalities. The unified communication system <NUM> can allow the message originator, when it is a control center, to send messages to each of a plurality of vehicle operators using the right modality for each vehicle operator. The unified communication system <NUM> is configured to automatically select a correct modality for a message transmission.

The example unified system <NUM> includes an automatic message generation module <NUM> and a recipient modality selection module <NUM>. The example process includes a message originator <NUM> entering a data message via an electronic device <NUM>, such as a terminal, tablet computer, EFB, smartphone, or other electronic device (operation <NUM>). The automatic message generation module <NUM>, in turn, automatically generates a data message for transmission using the proper modality for transmission.

The recipient modality selection module <NUM> is configured to determine the proper modality for transmission. To do so, the recipient modality selection module <NUM> is configured to determine for each specific message recipient the existence of any predetermined vehicle context requirements <NUM>, predetermined static requirements <NUM>, predetermined communication service provider requirements <NUM>, communication context requirements <NUM>, message recipient preferences <NUM>, and message originator preferences <NUM>. The recipient modality selection module <NUM> is configured to make a modality selection based on the existing predetermined vehicle context requirements <NUM>, predetermined static requirements <NUM>, predetermined communication service provider requirements <NUM>, communication context requirements <NUM>, message recipient preferences <NUM>, and message originator preferences <NUM>. Examples of a predetermined vehicle context requirement <NUM> includes the existence of a non-normal vehicle state such as fire on-board the vehicle or the vehicle being driven in a very reactive environment such as final approach or landing for an aircraft, and lots of turns for an automobile. The predetermined vehicle context requirement in these examples may necessitate the use of voice instead of data for a communication modality. An example of a predetermined static requirement <NUM> may be a requirement wherein the message recipient can only receive a message in less than all of the available modalities, such as the ability to only receive a digital data or analog voice message. An example of a predetermined communication service provider requirement <NUM> may be a requirement wherein the communication service provider only allows communication with a specific recipient using a particular modality such as digital data. An example of user preferences <NUM> may be a preference wherein the user may want to transmit certain types of messages only using one specific modality and transmit other types of messages using a different modality. An example of communication context requirements <NUM> may be a requirement wherein when the latest communication between the message originator and message recipient is via a first modality type, subsequent communication between the message originator and the message recipient should also be via that same first modality type.

After the message originator's entry of a data message, the process includes the automatic generation of a data message (operation <NUM>) and a determination of the destination for the data (decision <NUM>). The recipient modality selection module <NUM> will determine whether the modality should be data or voice and provide that information to the automatic message generation module <NUM>. At decision <NUM>, when it has been determined that the recipient modality should be data, the message is formatted by the automatic message generation module <NUM> as a data link message (operation <NUM>). When it has been determined that the recipient modality should be voice, the message is formatted by the automatic message generation module <NUM> as a digital voice message (operation <NUM>).

In this example, regardless of whether the message is formatted as digital voice or formatted as a data link message, the message is sent over a network such as an IP network to the message recipient (operation <NUM>). If the message is received by the message recipient <NUM> as digital voice, the message recipient <NUM> will receive the message, for example, via a digital telephone or digital phone software running on an electronic device <NUM>. If the message is received by the message recipient <NUM> as digital data, the message recipient <NUM> will receive the message via an electronic device <NUM>. The automatic message generation module <NUM> is also configured to maintain communication context (operation <NUM>). The automatic message generation module <NUM> can maintain communication context by communicating the last modality used to the recipient modality selection module <NUM> (operation <NUM>).

The example unified system <NUM> also includes a digital assistant <NUM> that is configured to automatically communicate in case the message recipient <NUM> is busy. When the digital assistant <NUM> has been activated by the message recipient <NUM>, the digital assistant <NUM> can appropriately respond to non-critical messages.

For example, a pilot (e.g., message recipient <NUM>) could activate automatic VHF frequency transfer. If a message commanding VHF frequency transfer is received from the ground (e.g., message originator <NUM>) via voice or via data, and if the pilot has activated the automatic communication assistant (e.g., digital assistant <NUM>), the digital assistant <NUM> will automatically perform the action of automatic VHF frequency transfer.

While communication between two entities is depicted, the example unified system <NUM> should not be limited to two entities. For example, one truck driver could communicate with several centers and several other truck drivers, but there will be communication context management for each pair of communicating entities, possibly using common factors (e.g. user preferences are the same for all Control Centers).

<FIG> is a data flow diagram depicting another example process in a unified communication system <NUM> for use by a message originator <NUM> for sending messages to one or more message recipients <NUM>. The message originator <NUM> may be a vehicle operator such as a pilot (or other flight crew member), a truck driver, or some other type of vehicle operator who may need to communicate with a plurality of different control centers (e.g., ATC) during a mission. Different modalities may be used to communicate with the different control centers. The unified communication system <NUM> is configured to automatically select a communication modality for use when communicating with a particular control center freeing the vehicle operator to focus his/her attention on matters other than choosing the correct modality for the particular control center.

Alternatively, the message originator <NUM> may be a control center that needs to communicate with a plurality of different vehicles that use different communication modalities. The unified communication system <NUM> can allow the message originator <NUM>, when it is a control center, to send messages to each of a plurality of vehicle operators using the right modality for each vehicle operator. The unified communication system <NUM> is configured to automatically select a correct modality for a message transmission.

The example unified system <NUM> includes an automatic message generation module <NUM> and a recipient modality selection module <NUM>. The example process includes a message originator <NUM> sending an analog voice message via an analog voice device <NUM>, such as a telephone (operation <NUM>). A voice recognition module <NUM> comprising voice recognition software, for example, may perform voice recognition operations to convert the analog voice received from the analog voice device <NUM> into digital voice and/or digital data. The automatic message generation module <NUM>, in turn, automatically generates a data message for transmission using the proper modality for transmission.

The recipient modality selection module <NUM> is configured to determine the proper modality for transmission. To do so, the recipient modality selection module <NUM> is configured to determine for each specific message recipient the existence of any predetermined vehicle context requirements <NUM>, predetermined static requirements <NUM>, predetermined communication service provider requirements <NUM>, communication context requirements <NUM>, message recipient preferences <NUM>, and message originator preferences <NUM>. The recipient modality selection module <NUM> is configured to make a modality selection based on the existing predetermined vehicle context requirements <NUM>, predetermined static requirements <NUM>, predetermined communication service provider requirements <NUM>, communication context requirements <NUM>, message recipient preferences <NUM>, and message originator preferences <NUM>. Examples of a predetermined vehicle context requirement <NUM> includes the existence of a non-normal vehicle state such as fire on-board the vehicle or the vehicle being driven in a very reactive environment such as final approach or landing for an aircraft, and lots of turns for an automobile. The predetermined vehicle context requirement in these examples may necessitate the use of voice instead of data for a communication modality. An example of a predetermined static requirement <NUM> may be a requirement wherein the message recipient can only receive less than all of the available modalities, such as can only receive a digital data or analog voice message. An example of a predetermined communication service provider requirement <NUM> may be a requirement wherein the communication service provider only allows communication with a specific recipient using a particular modality such as digital data. An example of user preferences <NUM> may be a preference wherein the user may want to transmit certain types of messages only using one specific modality and transmitting other types of messages using a different modality. An example of communication context requirements <NUM> may be a requirement wherein when the latest communication between the message originator and message recipient is via a first modality type, subsequent communication between the message originator and the message recipient should also be via that same first modality type.

After the message originator's entry of a voice message and the conversion of the voice message to digital voice and/or data, the process includes a determination of whether the modality for the message is digital data (decision <NUM>). The recipient modality selection module <NUM> will determine whether the modality should be data or voice and provide that information to the automatic message generation module <NUM>. At decision <NUM>, when it has been determined that the recipient modality should be data (yes at decision <NUM>), a decision is made regarding whether voice recognition confidence is sufficient for the data criticality (decision <NUM>). If the voice recognition confidence is not sufficient for the data criticality (no at decision <NUM>), the process includes informing the user (operation <NUM>). If the voice recognition confidence is indeed sufficient for the data criticality (yes at decision <NUM>), the process includes formatting by the automatic generation message module <NUM> the message as a data link message (operation <NUM>). When it has been determined that the recipient modality should be voice (no at decision <NUM>), the message is formatted by the automatic message generation module <NUM> as a digital voice message (operation <NUM>).

Similar systems are envisioned for a unified communication system when other modalities are used for message generation by the message originator and/or other modalities are used for the message that is transmitted to the message recipient. As illustrated in the following examples: (a) when the received communication is an analog voice communication and the selected modality is digital data transmission, the unified communication system can be configured to cause the analog voice communication to be converted to a digital data transmission using voice recognition software, determine whether voice recognition confidence is sufficient for the criticality of conveyed information, cause the digital data transmission to be transmitted to the message recipient when the voice recognition confidence is sufficient for the criticality of conveyed information, and notify the message originator when the voice recognition confidence is not sufficient; (b) when the received communication is an analog voice communication and the selected modality is digital voice transmission, the unified communication system can be configured to cause the analog voice communication to be converted to a digital voice transmission using voice recognition software, determine whether voice recognition confidence is sufficient for the criticality of conveyed information, cause the digital voice transmission to be transmitted to the message recipient when the voice recognition confidence is sufficient for the criticality of conveyed information, and notify the message originator when the voice recognition confidence is not sufficient; (c) when the received communication is digital text and the selected modality is digital data transmission, the unified communication system can be configured to cause the digital text to be converted to a digital data transmission and cause the digital data transmission to be transmitted to the message recipient; (d) when the received communication is digital text and the selected modality is digital voice transmission, the unified communication system can be configured to cause the digital text to be converted to a digital voice transmission and cause the digital voice transmission to be transmitted to the message recipient; (e) when the received communication is a digital data transmission and the selected modality is digital text, the unified communication system can be configured to cause the digital data transmission to be converted to digital text and cause the digital text to be transmitted to the message recipient; (f) when the received communication is a digital data transmission and the selected modality is an analog voice communication, the unified communication system can be configured to cause the digital data transmission to be converted to an analog voice communication and cause the analog voice communication to be transmitted to the message recipient; (g) when the received communication is a digital voice transmission and the selected modality is digital text, the unified communication system can be configured to cause the digital voice transmission to be converted to digital text and cause the digital text to be transmitted to the message recipient; and/or (h) when the received communication is a digital voice transmission and the selected modality is an analog voice communication, the unified communication system can be configured to cause the digital voice transmission to be converted to an analog voice communication and cause the analog voice communication to be transmitted to the message recipient.

<FIG> is a block diagram depicting an example unified communication system <NUM> for automatically choosing a communication modality from a plurality of communication modalities for use for communication between distant communication entities. The unified communication system <NUM> comprises a controller <NUM>. The example controller <NUM> includes at least one processor and a computer-readable storage device or media encoded with programming instructions for configuring the controller. The processor may be any custom-made or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), an auxiliary processor among several processors associated with the controller, a semiconductor-based microprocessor (in the form of a microchip or chip set), any combination thereof, or generally any device for executing instructions.

The computer readable storage device or media may include volatile and nonvolatile storage in read-only memory (ROM), random-access memory (RAM), and keep-alive memory (KAM), for example. KAM is a persistent or non-volatile memory that may be used to store various operating variables while the processor is powered down. The computer-readable storage device or media may be implemented using any of a number of known memory devices such as PROMs (programmable read-only memory), EPROMs (electrically PROM), EEPROMs (electrically erasable PROM), flash memory, or any other electric, magnetic, optical, or combination memory devices capable of storing data, some of which represent executable programming instructions, used by the controller.

The example controller <NUM> is configured to: receive a message <NUM> in the form of one of a first plurality of modalities from a message originator for transmission to a message recipient. The message originator may be one of a vehicle operator or a central control system and the message recipient may be the other of the vehicle operator or the central control system. Alternatively, both the message originator and the message operator may be a vehicle operator. The example controller <NUM> is configured to receive the communication from the message originator in the form of analog voice, digital text, digital voice transmission, a digital data transmission, and/or other forms.

The example controller <NUM> is configured to select at least one modality from a second plurality of modalities for sending the message to the message recipient, wherein the second plurality of modalities may include a digital data transmission, a digital voice transmission, a digital text message, an analog voice message, and/or others. To select, the example controller <NUM> may include a selection module <NUM> that is configured to: determine for the specific message recipient the existence of any predetermined vehicle context requirements, predetermined static requirements, predetermined communication service provider requirements, communication context requirements, message recipient preferences, message originator preferences and/or other criteria; and make the message context selection based on the existing predetermined vehicle context requirements, predetermined static requirements, predetermined communication service provider requirements, communication context requirements, message recipient preferences, message originator preferences and/or other criteria. The example controller <NUM> may include a conversion module <NUM> that is configured to convert the received message to the selected modality when the selected modality is different from the modality of the received message. The example controller <NUM> is further configured to cause the message <NUM> to be transmitted in the form of the selected modality to the message recipient.

To make the selection, the example controller <NUM> may be configured (e.g., via the selection module <NUM>) to make a selection that satisfies the predetermined vehicle context requirements, the predetermined static requirements and/or the predetermined communication service provider requirements. To make the selection, the example controller <NUM> may be configured to make the selection that satisfies communication context requirements, message recipient preference, and/or a message originator preference when the predetermined vehicle context requirements, the predetermined static requirements and predetermined communication service provider requirements are not dispositive for the selection. The communication context requirements may comprise the last modality used in communication between originator and recipient. The static requirements may comprise the only modality the recipient can use for message reception. The communication service provider requirements may comprise a requirement based on message transmission cost.

To convert the received message to the selected modality, the example controller <NUM> may be configured (e.g., via the conversion module <NUM>) to, when the received communication is an analog voice communication and the selected modality is digital data transmission, cause the analog voice communication to be converted to a digital data transmission using voice recognition software, determine whether voice recognition confidence is sufficient for the criticality of conveyed information, cause the digital data transmission to be transmitted to the message recipient when the voice recognition confidence is sufficient for the criticality of conveyed information, and notify the message originator when the voice recognition confidence is not sufficient.

To convert the received message to the selected modality the example controller <NUM> may be configured (e.g., via the conversion module <NUM>) to, when the received communication is an analog voice communication and the selected modality is digital voice transmission, cause the analog voice communication to be converted to a digital voice transmission using voice recognition software, determine whether voice recognition confidence is sufficient for the criticality of conveyed information, cause the digital voice transmission to be transmitted to the message recipient when the voice recognition confidence is sufficient for the criticality of conveyed information, and notify the message originator when the voice recognition confidence is not sufficient.

To convert the received message to the selected modality, the example controller <NUM> may be configured (e.g., via the conversion module <NUM>) to, when the received communication is digital text and the selected modality is digital data transmission, cause the digital text to be converted to a digital data transmission and cause the digital data transmission to be transmitted to the message recipient.

To convert the received message to the selected modality, the example controller <NUM> may be configured (e.g., via the conversion module <NUM>) to, when the received communication is digital text and the selected modality is digital voice transmission, cause the digital text to be converted to a digital voice transmission and cause the digital voice transmission to be transmitted to the message recipient.

To convert the received message to the selected modality, the example controller <NUM> may be configured (e.g., via the conversion module <NUM>) to, when the received communication is a digital data transmission and the selected modality is digital text, cause the digital data transmission to be converted to digital text and cause the digital text to be transmitted to the message recipient.

To convert the received message to the selected modality, the example controller <NUM> may be configured (e.g., via the conversion module <NUM>) to, when the received communication is a digital data transmission and the selected modality is an analog voice communication, cause the digital data transmission to be converted to an analog voice communication and cause the analog voice communication to be transmitted to the message recipient.

To convert the received message to the selected modality, the example controller <NUM> may be configured (e.g., via the conversion module <NUM>) to, when the received communication is a digital voice transmission and the selected modality is digital text, cause the digital voice transmission to be converted to digital text and cause the digital text to be transmitted to the message recipient.

To convert the received message to the selected modality the example controller <NUM> may be configured (e.g., via the conversion module <NUM>) to, when the received communication is a digital voice transmission and the selected modality is an analog voice communication, cause the digital voice transmission to be converted to an analog voice communication and cause the analog voice communication to be transmitted to the message recipient.

The transmission modality selection may depend on a number of factors. The transmission modality selection may depend on the availability of the modality at the destination. For example, if only analog voice is implemented at destination, only analog voice can be used as the transmission modality. For example, if datalink is implemented but there is a system component failure in the datalink chain of equipment, then datalink may not be used as the transmission modality. In another example, if datalink is implemented only for some messages, then datalink may not be used as the transmission modality for messages that do not meet the requirement for datalink transmission.

The transmission modality selection may depend on the cost of communication. For example, a contracted service vendor may prefer that data transmission be used instead of voice transmission because data transmissions may be more cost efficient than digitalized voice.

The transmission modality selection may depend on the system configuration at the destination: By design, voice might be considered for some messages (e.g., short obvious messages); and data might be required for messages concerning complex objects (e.g., provision of GPS coordinates).

The transmission modality selection may depend on user preference at the destination. For example, the message recipient may choose voice or data depending of the messages. In another example, if a message recipient first communicated by voice to the message originator, the transmission modality selection may be by voice for the reply communication from the message originator to the message recipient.

Transmission modality selection could be synchronized between the unified communication system operated by the message originator and a system operated by the message recipient. This could be accomplished by the exchange of supervision messages between both systems to communicate recipient preferences.

When multiple communications exist between the message originator and a particular message recipient, the unified communication system <NUM> may be configured to link those messages even when the multiple communications are sent using different modalities. The unified communication system <NUM> may be configured to determine if a voice message overrides a data message, showing clearly the obsolete status of a message to the receiving user. The unified communication system <NUM> may also be configured to determine if a data message overrides a voice message, showing clearly the obsolete status of the message to the receiving user.

<FIG> is a process flow chart depicting an example process <NUM> for automatically choosing a communication modality from a plurality of communication modalities for use for communication between distant communication entities. The order of operation within the process <NUM> is not limited to the sequential execution as illustrated in the figure, but may be performed in one or more varying orders as applicable and in accordance with the present disclosure.

The example process <NUM> includes receiving a message in the form of one of a first plurality of modalities from a message originator for transmission to a message recipient (operation <NUM>). The example process <NUM> may be used by a vehicle operator or a central control system. The message originator may be one of a vehicle operator or a central control system and the message recipient may be the other of the vehicle operator or the central control system. Alternatively, both the message originator and the message operator may be a vehicle operator.

The example process <NUM> includes selecting at least one modality from a second plurality of modalities for sending the message to the message recipient (operation <NUM>). The second plurality of modalities may include a plurality of digital data transmission, digital voice transmission, digital text message, an analog voice message, and others. The selecting may include: determining for the specific message recipient the existence of any predetermined vehicle context requirements, predetermined static requirements, predetermined communication service provider requirements, communication context requirements, message recipient preferences, and message originator preferences; and making the selection based on the existing predetermined vehicle context requirements, predetermined static requirements, predetermined communication service provider requirements, communication context requirements, message recipient preferences, and message originator preferences. The communication context requirements may include the last modality used in communication between originator and recipient. The static requirements may include the only modality the recipient can use for message reception (temporarily or permanently). The communication service provider requirements comprise a requirement based on message transmission cost.

Making the selection may comprise making a selection that satisfies the predetermined vehicle context requirements, the predetermined static requirements and/or the predetermined communication service provider requirements. Making the selection may comprise making a selection that satisfies communication context requirements, message recipient preference, and/or a message originator preference when the predetermined vehicle context requirements, the predetermined static requirements and predetermined communication service provider requirements are not dispositive for the selection.

The example process <NUM> includes converting the received message to the selected modality when the selected modality is different from the modality of the received message (operation <NUM>) and causing the message to be transmitted in the form of the selected modality to the message recipient (operation <NUM>).

Converting the received message to the selected modality when the selected modality is different from the modality of the received message may comprise: when the received communication is an analog voice communication and the selected modality is digital data transmission, causing the analog voice communication to be converted to a digital data transmission using voice recognition software, determining whether voice recognition confidence is sufficient for the criticality of conveyed information, causing the digital data transmission to be transmitted to the message recipient when the voice recognition confidence is sufficient for the criticality of conveyed information, and notifying the message originator when the voice recognition confidence is not sufficient.

Converting the received message to the selected modality when the selected modality is different from the modality of the received message may comprise: when the received communication is an analog voice communication and the selected modality is digital voice transmission, causing the analog voice communication to be converted to a digital voice transmission using voice recognition software, determining whether voice recognition confidence is sufficient for the criticality of conveyed information, causing the digital voice transmission to be transmitted to the message recipient when the voice recognition confidence is sufficient for the criticality of conveyed information, and notifying the message originator when the voice recognition confidence is not sufficient.

Converting the received message to the selected modality when the selected modality is different from the modality of the received message may comprise: when the received communication is digital text and the selected modality is digital data transmission, causing the digital text to be converted to a digital data transmission and causing the digital data transmission to be transmitted to the message recipient.

Converting the received message to the selected modality when the selected modality is different from the modality of the received message may comprise: when the received communication is digital text and the selected modality is digital voice transmission, causing the digital text to be converted to a digital voice transmission and causing the digital voice transmission to be transmitted to the message recipient.

Converting the received message to the selected modality when the selected modality is different from the modality of the received message may comprise: when the received communication is a digital data transmission and the selected modality is digital text, causing the digital data transmission to be converted to digital text and causing the digital text to be transmitted to the message recipient.

Converting the received message to the selected modality when the selected modality is different from the modality of the received message may comprise: when the received communication is a digital data transmission and the selected modality is an analog voice communication, causing the digital data transmission to be converted to an analog voice communication and causing the analog voice communication to be transmitted to the message recipient.

Converting the received message to the selected modality when the selected modality is different from the modality of the received message may comprise: when the received communication is a digital voice transmission and the selected modality is digital text, causing the digital voice transmission to be converted to digital text and causing the digital text to be transmitted to the message recipient.

Converting the received message to the selected modality when the selected modality is different from the modality of the received message may comprise: when the received communication is a digital voice transmission and the selected modality is an analog voice communication, causing the digital voice transmission to be converted to an analog voice communication and causing the analog voice communication to be transmitted to the message recipient.

Described herein are apparatus, systems, techniques and articles for automatically choosing a communication modality from a plurality of communication modalities for use for communication between distant communication entities. The apparatus, systems, techniques and articles provided herein can reduce the workload associated with driver (e.g., pilot) /controller communication. The apparatus, systems, techniques and articles provided herein can reduce the workload by allowing the communication users to choose at any time the best communication modality. The apparatus, systems, techniques and articles provided herein can reduce the workload by reducing analyzing time thanks to a unified representation of incoming voice and data messages in a consistent way. The apparatus, systems, techniques and articles provided herein can reduce the workload by enabling a more systematic automation of actions driven by messages. The apparatus, systems, techniques and articles provided herein can increase safety by avoiding confusion in communication. The apparatus, systems, techniques and articles provided herein can decrease operation costs by allowing the vehicle owner (and airline operating center) to optimize communication costs by selecting the most cost efficient mode of communication at a given time. The apparatus, systems, techniques and articles provided herein can decrease vehicle costs by allowing use of fully digital solutions for both data and voice (potential applicability to airborne mobility).

In one embodiment, a communication system for automatically choosing a communication modality from a plurality of communication modalities for use for communication between distant communication entities is provided. The communication system comprising a controller configured to receive a message in the form of one of a first plurality of modalities from a message originator for transmission to a message recipient. The controller is further configured to select at least one modality from a second plurality of modalities for sending the message to the message recipient, wherein the second plurality of modalities includes a digital data transmission, a digital voice transmission, a digital text message and an analog voice message, and wherein to select the controller is further configured to: determine for the specific message recipient the existence of any predetermined vehicle context requirements, predetermined static requirements, predetermined communication service provider requirements, communication context requirements, message recipient preferences, and message originator preferences; and make the selection based on the existing predetermined vehicle context requirements, predetermined static requirements, predetermined communication service provider requirements, communication context requirements, message recipient preferences, and message originator preferences. The controller is further configured to convert the received message to the selected modality when the selected modality is different from the modality of the received message and transmit the message in the form of the selected modality to the message recipient.

These aspects and other embodiments may include one or more of the following features. The message originator may be one of a vehicle operator or a central control system and the message recipient may be the other of the vehicle operator or the central control system. The controller may be configured to receive the communication from the message originator in the form of analog voice, digital text, digital voice transmission, and a digital data transmission. To make the selection, the controller may be configured to make a selection that satisfies the predetermined vehicle context requirements, the predetermined static requirements and/or the predetermined communication service provider requirements. To make the selection, the controller may be configured to make the selection that satisfies communication context requirements, message recipient preference, and/or a message originator preference when the predetermined vehicle context requirements, the predetermined static requirements and predetermined communication service provider requirements are not dispositive for the selection. The communication context requirements comprise the last modality used in communication between originator and recipient. The static requirements may comprise the only modality the recipient can use for message reception. The communication service provider requirements may comprise a requirement based on message transmission cost. To convert the received message to the selected modality, the controller may be configured to, when the received communication is an analog voice communication and the selected modality is digital data transmission, cause the analog voice communication to be converted to a digital data transmission using voice recognition software, determine whether voice recognition confidence is sufficient for the criticality of conveyed information, cause the digital data transmission to be transmitted to the message recipient when the voice recognition confidence is sufficient for the criticality of conveyed information, and notify the message originator when the voice recognition confidence is not sufficient. To convert the received message to the selected modality, the controller may be configured to, when the received communication is an analog voice communication and the selected modality is digital voice transmission, cause the analog voice communication to be converted to a digital voice transmission using voice recognition software, determine whether voice recognition confidence is sufficient for the criticality of conveyed information, cause the digital voice transmission to be transmitted to the message recipient when the voice recognition confidence is sufficient for the criticality of conveyed information, and notify the message originator when the voice recognition confidence is not sufficient. To convert the received message to the selected modality, the controller may be configured to, when the received communication is digital text and the selected modality is digital data transmission, cause the digital text to be converted to a digital data transmission and cause the digital data transmission to be transmitted to the message recipient. To convert the received message to the selected modality, the controller may be configured to, when the received communication is digital text and the selected modality is digital voice transmission, cause the digital text to be converted to a digital voice transmission and cause the digital voice transmission to be transmitted to the message recipient. To convert the received message to the selected modality, the controller may be configured to, when the received communication is a digital data transmission and the selected modality is digital text, cause the digital data transmission to be converted to digital text and cause the digital text to be transmitted to the message recipient. To convert the received message to the selected modality, the controller may be configured to, when the received communication is a digital data transmission and the selected modality is an analog voice communication, cause the digital data transmission to be converted to an analog voice communication and cause the analog voice communication to be transmitted to the message recipient. To convert the received message to the selected modality, the controller may be configured to, when the received communication is a digital voice transmission and the selected modality is digital text, cause the digital voice transmission to be converted to digital text and cause the digital text to be transmitted to the message recipient. When the received communication is a digital voice transmission and the selected modality is an analog voice communication, to convert the received message to the selected modality, the controller may be configured to cause the digital voice transmission to be converted to an analog voice communication and cause the analog voice communication to be transmitted to the message recipient.

In another embodiment, a processor-implemented method for automatically choosing a communication modality from a plurality of communication modalities for use for communication between distant communication entities is provided. The method comprises: receiving a message in the form of one of a first plurality of modalities from a message originator for transmission to a message recipient. The method further comprises selecting at least one modality from a second plurality of modalities for sending the message to the message recipient, wherein the second plurality of modalities includes a digital data transmission, a digital voice transmission, a digital text message and an analog voice message, and wherein the selecting comprises: determining for the specific message recipient the existence of any predetermined vehicle context requirements, predetermined static requirements, predetermined communication service provider requirements, communication context requirements, message recipient preferences, and message originator preferences; and making the selection based on the existing predetermined vehicle context requirements, predetermined static requirements, predetermined communication service provider requirements, communication context requirements, message recipient preferences, and message originator preferences. The method further comprises converting the received message to the selected modality when the selected modality is different from the modality of the received message; and transmitting the message in the form of the selected modality to the message recipient.

These aspects and other embodiments may include one or more of the following features. The message originator may be one of a vehicle operator or a central control system and the message recipient may be the other of the vehicle operator or the central control system. Making the selection may comprise making a selection that satisfies the predetermined vehicle context requirements, the predetermined static requirements and/or the predetermined communication service provider requirements. Making the selection may comprise making a selection that satisfies communication context requirements, message recipient preference, and/or a message originator preference when the predetermined vehicle context requirements, the predetermined static requirements and predetermined communication service provider requirements are not dispositive for the selection. Converting the received message to the selected modality when the selected modality is different from the modality of the received message may comprise: when the received communication is an analog voice communication and the selected modality is digital data transmission, causing the analog voice communication to be converted to a digital data transmission using voice recognition software, determining whether voice recognition confidence is sufficient for the criticality of conveyed information, causing the digital data transmission to be transmitted to the message recipient when the voice recognition confidence is sufficient for the criticality of conveyed information, and notifying the message originator when the voice recognition confidence is not sufficient; when the received communication is an analog voice communication and the selected modality is digital voice transmission, causing the analog voice communication to be converted to a digital voice transmission using voice recognition software, determining whether voice recognition confidence is sufficient for the criticality of conveyed information, causing the digital voice transmission to be transmitted to the message recipient when the voice recognition confidence is sufficient for the criticality of conveyed information, and notifying the message originator when the voice recognition confidence is not sufficient; when the received communication is digital text and the selected modality is digital data transmission, causing the digital text to be converted to a digital data transmission and causing the digital data transmission to be transmitted to the message recipient; when the received communication is digital text and the selected modality is digital voice transmission, causing the digital text to be converted to a digital voice transmission and causing the digital voice transmission to be transmitted to the message recipient; when the received communication is a digital data transmission and the selected modality is digital text, causing the digital data transmission to be converted to digital text and causing the digital text to be transmitted to the message recipient; when the received communication is a digital data transmission and the selected modality is an analog voice communication, causing the digital data transmission to be converted to an analog voice communication and causing the analog voice communication to be transmitted to the message recipient; when the received communication is a digital voice transmission and the selected modality is digital text, causing the digital voice transmission to be converted to digital text and causing the digital text to be transmitted to the message recipient; and/or when the received communication is a digital voice transmission and the selected modality is an analog voice communication, causing the digital voice transmission to be converted to an analog voice communication and causing the analog voice communication to be transmitted to the message recipient.

In another embodiment, non-transitory computer readable media encoded with programming instructions configurable to cause a processor to perform a method is provided. The method comprises: receiving a message in the form of one of a first plurality of modalities from a message originator for transmission to a message recipient. The method further comprises selecting at least one modality from a second plurality of modalities for sending the message to the message recipient, wherein the second plurality of modalities includes a digital data transmission, a digital voice transmission, a digital text message and an analog voice message, and wherein the selecting comprises: determining for the specific message recipient the existence of any predetermined vehicle context requirements, predetermined static requirements, predetermined communication service provider requirements, communication context requirements, message recipient preferences, and message originator preferences; and making the selection based on the existing predetermined vehicle context requirements, predetermined static requirements, predetermined communication service provider requirements, communication context requirements, message recipient preferences, and message originator preferences. The method further comprises converting the received message to the selected modality when the selected modality is different from the modality of the received message; and transmitting the message in the form of the selected modality to the message recipient.

Claim 1:
An aircraft communication system for automatically choosing a communication modality from a plurality of communication modalities for use for communication between distant communication entities, the communication system comprising a controller configured to:
receive a communication in the form of an analog or digital voice message from a message originator, the message originator being a member of a flight crew of the aircraft, for transmission to a message recipient (<NUM>), the message recipient being an aircraft control center;
select, using a recipient modality selection module (<NUM>) at least one modality from a plurality of modalities for sending the analog or digital voice message to the message recipient, the plurality of modalities including a digital data transmission, a digital voice transmission, a digital text message and an analog voice message, wherein, in order to select the at least one modality, the controller is further configured to:
determine for the existence of a predetermined aerial vehicle context requirements indicative of a non-normal vehicle state, predetermined static requirements, predetermined communication service provider requirements, communication context requirements, message recipient preferences, and message originator preferences; and
make the selection based on the determined predetermined aerial vehicle context requirements, predetermined static requirements, predetermined communication service provider requirements, communication context requirements, message recipient preferences, and message originator preferences;
convert the analog or digital voice message to the selected modality, the selected modality being different from the modality of the analog or digital voice message; and
cause the received analog or digital voice message to be transmitted in the form of the selected modality to the message recipient, wherein the controller is further configured to communicate the selected modality to the recipient modality selection module (<NUM>).