Patent Description:
An example of the limited fidelity of a VR game engine is that a control panel in a virtual environment is often displayed as a polygon having a texture bitmapped onto the polygon so that it resembles a physical control panel, but yet does not provide a realistic interactive experience across a full range of potential inputs. In some situations, the texture bitmaps are static, or the simulated control panel has only limited functionality. Thus, a need exists for more realistic virtual reality with virtualization in trainers and test environments.

<CIT> describes, in accordance with its abstract, an external method for addressing the trainer-unique functions in a flight training system that allows a Control Display Navigation Unit (CDNU), loaded with an actual Operational Flight Program (OFP), to function properly in a Flight Simulator environment of a trainer system without sacrificing the trainer-unique functions. A CDNU Trainer Interface Unit (CTIU) is defined to realize the external method. The CTIU is installed between the CDNU and the trainer system. The communication between the CDNU with an OFP and the trainer system is thus controlled by the CTIU. The trainer-unique functions are controlled externally by the CTIU, thus eliminating an OTP in the CDNU. The CTIU connects the CDNU and the trainer system. It communicates with both the trainer system and the CDNU/OFP using two sets of MIL-STD-1553B Buses. To the CDNU/OFP, it acts like the real navigation and communication systems. To the trainer system, it acts like a CDNU OTP. The CTIU will be able to address the trainer-unique functions with no modification to the OFP and CDNU hardware. The functions of the Operational Training Program (OTP) will be realized by the CTIU external to the CDNU/OFP. The CTIU activates OTP-equivalent capabilities upon receipt of OTP active commands from the trainer system.

<CIT> describes, in accordance with its abstract, a method for in-vehicle dynamic virtual reality that includes receiving vehicle data from one or more vehicle systems of a vehicle, wherein the vehicle data includes vehicle dynamics data and receiving user data from a virtual reality device. The method includes generating a virtual view based on the vehicle data, the user data and a virtual world model, the virtual world model including one or more components that define the virtual view, wherein generating the virtual view includes augmenting one or more components of the virtual world model according to at least one of the vehicle data and the user data and rendering the virtual view to an output device by controlling the output device to update display of the virtual view according to the vehicle dynamics data.

<NPL> describes, in accordance with its abstract, flight simulators that are used for different purposes, such as pilot training, aircraft design and development. Full-scale flight simulators have high costs and dependency on aircraft type due to hardware constraints. Hence, virtual reality flight simulators are designed. On the other hand, they are generally created only for specific applications, such as helicopter simulators. As a result, these tools can hardly be used as a generic tool which can work with various aircraft simulations. Although, there are certain generic virtual reality applications that can be used for virtual prototyping and ergonomics, they lack realistic flight simulation and environment. A generic aerospace application is presented which brings a solution to these problems. The architecture of the application is described and a calibration method which, makes the application independent of the physical mock-up and the flight simulator compatible with different aircraft types, is presented. The preliminary results of the first prototype are given as the generic virtual reality flight simulator is used by the aerospace industry for research and development purposes.

<NPL> describes, in accordance with its abstract, research that presents a Virtual Reality Flight Simulator (VRFS) that combines the advantages of desktop simulations and hardware mock-ups, i.e. the flexibility of a desktop flight simulation with the level of immersion close to a full flight simulator. In contrast to similar existing VR flight simulators, the presented system focuses on Human Factors (HF) research and is used for evaluating flight decks already in an early phase of the design process. Four user studies are presented that demonstrate the application of integrated HF methods and the usability of the system.

<NPL>, describes, in accordance with its abstract, an unmanned aircraft system to coordinate a team of small aerial vehicles in area coverage monitoring, applied for persistent surveillance missions. The proposed system architecture is based on the coordination of embedded robotic agents, creating a flying ad hoc network composed of three modules: (a) ground control station, for mission planning and control through user interaction; (b) quadrotors team, capable of navigation by inertial sensors and global navigation satellite system; and, (c) control agents embedded in aircrafts, responsible for communications, task allocation and payload mission control.

The invention and preferred embodiments are defined in the appended claims.

In an aspect, a virtual reality, VR, environment platform in accordance with claim <NUM> is provided. In a further aspect, a method of operating an immersive virtual reality, VR, environment platform in accordance with claim <NUM> is provided.

Some aspects and examples disclosed herein are directed to virtual reality (VR) aircraft test and training environments that simultaneously leverage a high quality immersive environment engine (possibly a VR game engine) and an operational flight program (OFP) running on a virtual flight management computer (FMC) implemented on a virtual machine by using a communication channel that couples the immersive VR environment engine with the virtual FMC. Existing investment in flight simulators, test environment components, and any of navigation simulation, data link simulation, air traffic control simulation, and flight visualization modules can be advantageously employed to provide high quality, realistic testing and training capability.

Some aspects and examples disclosed herein are directed to a VR environment platform with an immersive VR environment engine having aircraft control and display units and providing an immersive VR environment of an aircraft; a virtual FMC implemented on a virtual machine running an OFP; and a communication channel coupling the immersive VR environment engine with the virtual FMC, the communication channel emulating FMC communication protocols to permit the immersive VR environment engine to run the virtual FMC.

References made throughout this disclosure relating to specific examples and implementations are provided solely for illustrative purposes but, unless indicated to the contrary, are not meant to limit all examples.

In contrast with game engines, full-featured flight simulators have high fidelity models of hardware such as flight management computers (FMC) that communicate with a multipurpose computer display unit (MCDU), also known as a multipurpose control and display unit). In order to achieve the desired level of realism, the MCDU and FMCs are often simulated with intricate models or achieved through the use of actual flight hardware or proxy flight hardware. However, these options are costly to produce and are reliant on the skills of the developer. Meanwhile, existing low level training devices are model-based and sometime supplemented with physical structures that mock up the flight deck, flight deck, ground station, etc. When present, these physical mock ups are close to the layout of the actual device being trained for, but may not match the actual layout in most other cases. Additionally, the subsystem models do not have as much fidelity as virtual machines (VMs). A model is typically created by looking at the input/output for a subsystem and creating software to mimic the behavior of this subsystem from an input/output perspective.

VMs also generate input/output behavior, thus may appear to be similar, although for VMs, the behavior is driven by actual unmodified or minimally modified flight software binary executables, such as an operational flight program (OFP). For example, the FMC OFP can be run on the VM. VMs, that run the FMC's OFP, also referred to herein as a "virtual FMC," can provide considerably higher fidelity in a training environment, compared with simpler models and can also provide confidence that the behavior observed in the trainer will be the same as the physical device that the trainer is representing. It may then be possible to reduce the need for expensive full motion trainers (often costing in excess of $<NUM>), by instead using a lower-cost compelling, immersive, high fidelity training platform.

Accordingly, examples of the present disclosure provide for virtual reality (VR) aircraft test and training environments that simultaneously leverage a high quality immersive environment engine (possibly a VR game engine) and an OFP running on a virtual FMC by using a communication channel that couples the immersive VR environment engine with the virtual FMC. Existing investment in flight simulators, test environment components, and any of navigation simulation, data link simulation, air traffic control simulation, and flight visualization modules can be advantageously employed to provide high quality, realistic testing and training capability.

Recent improvements in computer processing capabilities and advancements in VR and virtualization technologies are leveraged to create a new kind of training environment. A front end VR world provides an immersive and realistic visual experience for users, while the combinations of VR and real-time computer graphics allows users to perceive and interact with a virtual world in the same way they would with real devices. The integration of the VR world with a high fidelity back end consisting of models and VMs provides dynamic behavior that is nearly equivalent to physical hardware. The VMs allow for the actual unmodified or minimally modified flight software binaries and databases to be used in the training device, providing confidence that what is observed in the trainer will be the same as the actual aircraft. Additionally, testing environments are feasible with early creation of human-in-the-loop testing. Since the flight deck is created in the VR world, the flight deck can be prototyped and modified relatively inexpensively, and ultimately used as a design input for a physical flight deck. In general, when requirements are developed for the physical systems, models and virtual machines can be created and tested relatively quickly.

Various examples solve one or more problems related to the use of different protocols by different FMCs / MCDU's to communicate (e.g., ARINC <NUM>, MIL-STD-<NUM>, Ethernet). And, the messages can be packaged differently depending on the remote terminal (RT) and aircraft type. One or more examples disclosed herein use communication channels that allow for a uniform way to pack/unpack the messages. As a result, the communication channels can be incorporated in different processing engines, for example, a game engine, such as Unreal Engine <NUM>. As such, with the one or more examples of the present disclosure, coding is performed once, and can be used on different FMCs with fewer changes, and more consistency. This reduces errors / debugging time and expense. Examples of various embodiments will now be described.

It should be appreciated that while the various examples described herein relate to an aircraft environment, the present disclosure can be implemented in other environments. For example, the herein described examples can be implemented in connection with any complex system that requires training, such as spacecraft, cars, boats, medical devices, etc. Thus, the various components described herein are operable in many other systems.

<FIG> is a block diagram illustrating an exemplary VR environment platform <NUM> for providing a test environment in accordance with aspects of the present disclosure. VR environment platform <NUM> includes an immersive VR environment engine <NUM> comprising aircraft control and display units and providing an immersive VR environment of an aircraft flight deck. For example, immersive VR environment engine <NUM> includes an MCDU <NUM>, a mode control panel (MCP) <NUM>, a display system (DS) <NUM>, such as a MAX ® display system (MDS) available from Rockwell Collins of Cedar Rapids, Iowa, a radio tuning panel (RTP) <NUM>, an overhead panel <NUM>, and an electronic flight instrument system (EFIS)<NUM>. The MCDU <NUM> provides for navigation planning, and can display an electronic compass and flight information. The MCP <NUM> displays information such as airspeed and warnings for the flight crew. The DS <NUM> displays information such as the Primary Flight Display (PFD), Navigation Display (ND), and systems status. The overhead panel <NUM> provides interaction points, such as switches. In the VR environment, the overhead panel <NUM> can receive aircraft flight deck control inputs through virtual interaction points. The EFIS <NUM> is a (virtual version of a) flight deck instrument display system that displays flight data electronically rather than electromechanically. In some examples, the EFIS <NUM> is incorporated into the DS <NUM>.

A virtualization system <NUM> comprises one or more computer storage devices having computer-executable instructions stored thereon for operating the VR environment platform <NUM>, which, on execution by a computer, cause the computer to perform operations including operating the immersive VR environment engine <NUM>. A physical interaction point <NUM>, coupled to the virtualization system <NUM> and illustrated as a control stick, simulates an aircraft control and provides physical feedback to a user <NUM> of the VR environment platform <NUM>. It should be understood that other physical interaction points can be provided, for example, rudder pedals that also simulate an aircraft control and provide physical feedback to the user <NUM>. Also shown as coupled to the virtualization system <NUM> is a head mounted display (HMD) <NUM>, worn by the user <NUM>, for displaying a view of an immersive VR environment (as generated by the VR environment platform <NUM>) to the user <NUM>.

In the VR environment platform <NUM>, as shown in <FIG>, a test environment core component <NUM> interfaces with a flight simulator component <NUM>, the immersive VR environment engine <NUM>, and a virtual FMC <NUM>. The test environment core component <NUM> further interfaces with a navigation simulation <NUM>, a data link simulation <NUM>, an air traffic control simulation <NUM>, and a flight visualization <NUM>, in order to provide additional realism for the operation of the aircraft flight simulator component <NUM>. In other examples, other arrangements are possible with more or fewer components as necessary for the desired level of fidelity. In operation, the test environment core component <NUM> can be used, for example, to interject simulated failures and monitor subsystem operations. The test environment core component <NUM> is able to accomplish these tasks by the signaling among each of the flight simulator component <NUM>, the virtual FMC <NUM>, the navigation simulation <NUM>, the data link simulation <NUM>, the air traffic control simulation <NUM>, and the flight visualization <NUM> passing through the test environment core component <NUM>. Also shown as interfacing with the test environment core component <NUM> is a file server <NUM>, for storing data used by and generated by operations of the VR environment platform <NUM>.

In some examples, the flight simulator component <NUM> is the same binary executable that is used in high-end flight simulators. As such, the flight simulator component <NUM> can be interchangeable based on the configuration of the system. For example, the flight simulator component <NUM> can be configured to operate with or without a motion base (not shown), as needed for the desired level of fidelity. In the VR environment platform <NUM>, the flight simulator component <NUM> interfaces with the virtual FMC <NUM> through the test environment core component <NUM>, so that the test environment core component <NUM> can monitor the data traffic and inject simulated failures. The virtual FMC <NUM> runs an OFP, which, in some examples, is a copy of the same binary executable that is used in hardware FMCs on operational aircraft. The above-described arrangement provides a level of realism otherwise unattainable by typical game engines. The use of a communication channel, as described herein, also allows a game engine, in various examples, to interact with an OFP more easily than traditionally achievable by a game engine alone. The communication channel <NUM> couples the immersive VR environment engine <NUM> with the virtual FMC <NUM> through the test environment core component <NUM>. That is, data traffic is routed through the test environment core component <NUM>, which allows for example, test scripts and manual interaction to override nominal system behavior for testing purposes. The communication channel <NUM> emulates FMC communication protocols to permit the immersive VR environment engine <NUM> to interact with the virtual FMC <NUM> using the actual packed input/output (I/O) of the aircraft. In various examples, the communication channel <NUM> (with emulated FMC protocols) is also between the data link simulation <NUM> and the test environment core component <NUM>, and the air traffic control simulation <NUM> and the test environment core component <NUM>. The remaining connections illustrated in <FIG> do not use FMC communication protocols, and interconnect components that support the primary aircraft simulations and VMs. In some examples, the communication channel <NUM> is implemented using protocol buffers as known in the data communication technology. However, any suitable type of communication channel can be used.

The communication channel <NUM> supports any kind of packed input/output (I/O) to an aircraft avionics box, such as ARINC <NUM>, ARINC <NUM>, CAN bus, RS-<NUM>, MIL-STD-<NUM>, MIL-STD-<NUM>, ACARS, ATN, Ethernet, and so on. The I/O is transferred over the communication channel <NUM> using a form of interprocess communications if the link is virtual, e.g. sockets and UDP packets with aircraft I/O encapsulated in the packets. As another example, the packed I/O is transferred over physical I/O links by installing an ARINC <NUM> I/O card in a PC. The I/O is not stored in a file, but is dynamic information being shared between systems, just like in a real aircraft.

<FIG> is a block diagram illustrating an exemplary VR environment platform <NUM> for providing a training environment in accordance with aspects of the present disclosure. The VR environment platform <NUM> shares some components with the VR environment platform <NUM> of <FIG>, although whereas the VR environment platform <NUM> is used for testing, the VR environment platform <NUM> is used for training, such as training flight crews. In the VR environment platform <NUM>, the immersive VR environment engine <NUM> interfaces with the flight simulator component <NUM> and the virtual FMC <NUM>, directly. Thus, a communication channel <NUM> couples the immersive VR environment engine <NUM> with the virtual FMC <NUM>. The communication channel <NUM> emulates FMC communication protocols to permit the immersive VR environment engine <NUM> to interact with the virtual FMC <NUM>. In some configuration of <FIG>, the aircraft flight simulator <NUM> is located and connected to the other components similar to <FIG>.

Additionally, the flight simulator component <NUM> and the virtual FMC <NUM> are coupled without the test environment core component <NUM> as an intermediary, although in some examples, a component similar to the test environment core component <NUM> can be used. The flight simulator component <NUM> interfaces with both the immersive VR environment engine <NUM> and the virtual FMC <NUM>. The file server <NUM> is shown as coupled to the virtual FMC <NUM>, although in some examples, the file server <NUM> can be coupled to the flight simulator component <NUM>. Additionally, since the VR environment platform <NUM> is used for training, an instructor operator station component <NUM> interfaces with the flight simulator component <NUM>. The instructor operator station component <NUM> permits an instructor to monitor and possibly participate in training activities conducted with the VR environment platform <NUM>.

The VR environment platform <NUM> can be used as a flight management system (FMS) trainer. In some examples, a two-dimensional (2D) tablet-based application facilitates learning and practicing the operation of the MCDU <NUM>, the MCP <NUM>, and other displays (e.g., the DS <NUM>) prior to and during various phases of flight. Students can learn and to practice FMS data entry (flight plan, etc.), as well as autoflight and flight path management as part of an instructor-led curriculum or for standalone self-guided training.

The VR environment platform <NUM> can be used as an interactive, fully-simulated virtual aircraft flight deck. Students will be able to learn and practice procedures in a fully immersive realistic virtual environment as part of an instructor-led curriculum or for standalone self-guided training. 3D interactive VR applications with high fidelity components as described in the VR environment platform <NUM> have the potential to greatly reduce the cost of training, as well as improve the learning outcome through immersive experiences. It can also enable self and distance learning, reducing and potentially eliminating the need to travel to an onsite training location.

<FIG> is an illustration of the exemplary virtual FMC <NUM>, which can be executed as a virtual machine (VM) on the virtualization system <NUM>, shown in more detail in <FIG>. As illustrated, the virtual FMC <NUM> comprises a processor <NUM>, which can be a PowerPC processor, an ARINC bus <NUM> (e.g., ARINC <NUM> or ARINC <NUM>), a universal asynchronous receiver-transmitter (UART) <NUM>, an Ethernet module <NUM>, and an OFP <NUM>. The DITS <NUM> is the predominant avionics data bus used on many higher-end commercial and transport aircraft. It defines the physical and electrical interfaces of a two-wire data bus and a data protocol to support an aircraft's avionics local area network The UART <NUM> is a device for asynchronous serial communication in which the data format and transmission speeds are configurable.

The OFP <NUM> is the embedded software that runs on an avionics device, namely, the hardware FMC. The physical platform that the test environment can be used to develop and that the trainer is used to train to operate, as described herein, emulates an aircraft flight deck <NUM> as shown in <FIG>. That is, various examples emulate a physical system corresponding to the aircraft flight deck <NUM>.

The OFP <NUM> contains software logic that perform functions necessary for the operation of the FMC. In the example of the hardware FMC, the OFP <NUM> performs the route planning, calculations, logic, input/out, and the like. The OFP <NUM> typically executes on top of an operating system software (e.g., an embedded real-time operating system) and functional components. The OFP <NUM> can be specifically tailored to the hardware of the avionics, the aircraft, and the aircraft type. The hardware FMC OFP communicates with other aircraft systems using Avionics buses, such as ARINC <NUM>, examples of other aircraft systems include MCDU <NUM>, MCP <NUM>, DS <NUM>, Overhead Panel <NUM>, and EFIS <NUM>. It should be appreciated that protocol buffers are not typically used for communication on an actual aircraft. Standard Ethernet for which the protocol buffers are designed lack the guaranteed message delivery required for aircraft system operation.

<FIG> is a block diagram illustrating additional detail for the exemplary virtualization system <NUM>. The virtualization system <NUM> can be a computing system comprised of hardware and software that can emulate other devices. In implementations, the virtualization system <NUM> can be one or more general purpose computers, such as a server or a desktop computer that hosts one or more VMs that emulate physical hardware systems, such as a hardware FMC, as well as their hardware and software interfaces (e.g., peripherals, data links, interrupt behavior, and timing requirements). Additionally, a VM, such as the virtual FMC <NUM> execute an exact or substantially exact copy (e.g., an image) of the application software executed by the corresponding physical hardware system (for example the OFP <NUM>). In implementations, the virtualization system <NUM> can include, e.g., a hypervisor or virtual machine monitor software. For example, the virtualization system <NUM> can use QUICK EMULATOR ("QEMU"), which is an open source software application that performs hardware virtualization. Thus, the virtual FMC <NUM> can be an emulation of a physical hardware FMC within the virtualization system <NUM>.

The virtualization system <NUM> includes a computing device <NUM>, an input/output (I/O) device <NUM>, and a storage device <NUM>. The I/O device <NUM> can include any device that enables an individual to interact with the computing device <NUM> (e.g., a user interface) and/or any device that enables the computing device <NUM> to communicate with one or more other computing devices using any type of communications link. The I/O device <NUM> can be, for example, a touchscreen display, pointer device, keyboard, etc..

The storage device <NUM> can comprise a computer-readable, non-volatile hardware storage device that stores information and program instructions. For example, the storage device <NUM> can be one or more flash drives and/or hard disk drives. In accordance with aspects of the present disclosure, the storage device <NUM> can store hardware system program code <NUM>, one or more memory maps <NUM>, interrupt logic <NUM>, and a device library <NUM>. The program code <NUM> can be application software of a physical hardware system (e.g., a physical FMC). In implementations, the program code <NUM> substantially mirrors that of the physical hardware system. The memory map <NUM> describes connections between components of the physical hardware systems from a memory interface perspective. For example, the memory map <NUM> can represent locations of information for memory registers of hardware components of the physical hardware system as an offset from a starting memory address. The interrupt logic <NUM> can be information describing the interrupt functionality of the physical hardware system, as detailed below. The device library <NUM> can be a repository of computer-readable information and instructions describing emulations of one or more physical hardware systems, which can be previously created and stored for future use.

In some examples, the computing device <NUM> includes one or more processors <NUM> (e.g., microprocessor, microchip, or application-specific integrated circuit), one or more memory devices <NUM> (e.g., RAM, read-only memory (ROM)), one or more I/O interfaces <NUM>, and one or more network interface devices <NUM>. The memory devices <NUM> can include a local memory (e.g., a random access memory and a cache memory) employed during execution of program instructions. Additionally, the computing device <NUM> includes at least one communication channel <NUM> (e.g., a data bus) by which it communicates with the I/O device <NUM> and the storage device <NUM>. The processor <NUM> executes computer program instructions (e.g., an operating system and/or application programs), which can be stored in the memory device <NUM> and/or storage device <NUM>.

The processor <NUM> can also execute computer program instructions of a virtualization application <NUM> (e.g., QEMU) and test and/or evaluation software <NUM>. The virtualization application <NUM> can be the same or similar to that previously described. For example, the virtualization application <NUM> can include a hypervisor or VM monitor software. In accordance with aspects of the present disclosure, the virtualization application <NUM> can provide a VM (e.g., the virtual FMC <NUM>) using the program code <NUM>, the memory map <NUM>, the interrupt logic <NUM>, and/or the device library <NUM>.

The training, test and/or evaluation software <NUM> can be an application or program including computer-readable instructions and information configured to train, test, evaluate, and/or validate software. For example, training, test and/or evaluation software <NUM> can execute test routines that verify program code of a hardware device behaves as expected in response to a predetermined scenario. Additionally, the training, test and evaluation software <NUM> can execute cybersecurity routines that test attack vectors of malicious software. For example, the training, test and/or evaluation software <NUM> can record complete state (registers, memory, hardware states, etc.) of a virtual machine as instructions execute to allow observation and analysis of a compromised system. The training, test and/or evaluation software <NUM> can be used to provide a trainer for users of the VR environment platform <NUM> (e.g. flight crew or maintenance trainers).

It is noted that the computing device <NUM> is representative of various possible equivalent-computing devices that can perform the processes described herein. To this extent, in embodiments, the functionality provided by the computing device <NUM> can be any combination of general and/or specific purpose hardware and/or computer program instructions. In the disclosed embodiments, the program instructions and hardware can be created using standard programming and engineering techniques, respectively.

<FIG> is a flow diagram <NUM> illustrating an exemplary process of providing an immersive VR environment with virtualization in trainers and test environments. Operation <NUM> includes initiating the immersive VR environment engine <NUM> comprising aircraft control and display units (e.g., the MCDU <NUM>, the MCP <NUM>, the DS <NUM>, the overhead panel <NUM>, and the electronic flight instrument system (EFIS) <NUM> and operation <NUM> includes providing an immersive VR environment of an aircraft. In some examples, a startup procedure for the system includes initiating a startup on the test environment core GUI (e.g., a user clicking "Startup"). However, other startup procedures in the VR environment technology can be used.

Operation <NUM> includes operating the flight simulator component <NUM> (configured with XML and JSON files) interfacing with the immersive VR environment engine <NUM> and the virtual FMC <NUM>. Operation <NUM> includes operating the virtual FMC <NUM> running the OFP <NUM>. The OFP <NUM> runs within the virtual FMC <NUM> on the virtualization system <NUM> in operation <NUM>. The configuration of the VM in various examples operates similar to physical aircraft hardware. For example, the flight software OFP is configured with operation program configuration (OPC) databases, and airline modifiable information (AMI), which are XML based in some configurations. It should be noted that in various examples, the communication channels are defined during design and implemented as compiled code.

Operation <NUM> includes coupling the immersive VR environment engine <NUM> with the virtual FMC <NUM> with a communication channels (e.g., one or more of the communication channels <NUM> and <NUM>). Operation <NUM> includes the communication channels (e.g., one or more of the communication channels <NUM> and <NUM>) emulating FMC communication protocols to permit the immersive VR environment engine <NUM> to run in parallel to and communicates with the virtual FMC <NUM> in operation <NUM>. Operation <NUM> includes displaying a view of the immersive VR environment. The display can be 2D or 3D, and can be on any suitable display device, such as a screen or an HMD, such as the HMD <NUM>. Operation <NUM> includes receiving aircraft cockpit control inputs for the immersive VR environment through virtual interaction points (e.g., the MCDU <NUM>, the MCP <NUM>, the DS <NUM>, the RTP <NUM>, the overhead panel <NUM>, and the EFIS <NUM>, which are simulations within the immersive VR environment engine <NUM>). Operation <NUM> includes providing physical feedback to a user of the VR environment platform through a physical interaction point simulating an aircraft control (e.g., the physical interaction point <NUM>). For example, a physical manifestation of a control stick and rudder pedals can provide mechanical resistance or move as controlled by flight simulation activities.

Decision <NUM> indicates two options: a test environment, such as the VR environment platform <NUM> of <FIG>, or a training environment, such as the VR environment platform <NUM> of <FIG>. If a training environment, then operation <NUM> includes operating the instructor operator station component <NUM> interfacing with the flight simulator component <NUM>. If a test environment, then operation <NUM> includes operating the test environment core component <NUM> interfacing with the flight simulator component <NUM>, the immersive VR environment engine <NUM>, and the virtual FMC <NUM>. In operation <NUM>, at least one of navigation simulation <NUM>, the data link simulation <NUM>, the air traffic control simulation <NUM>, and the flight visualization <NUM> interfacing with the test environment core component <NUM>.

<FIG> is an illustration of server consolidation in accordance with aspects of the present disclosure. In <FIG>, each of servers <NUM>, <NUM>, and <NUM> operates as one of VMs <NUM>, <NUM>, and <NUM>, respectively, on a single host server <NUM>. Although three servers (<NUM>, <NUM>, and <NUM>) are illustrated, it should be understood that a different number of servers can be combined as VMs on a host server. A hypervisor <NUM> manages the execution of the VMs <NUM>, <NUM>, and <NUM> as guest machines. Operation in this manner permits the various components of the VR environment platform <NUM> of <FIG> and the VR environment platform <NUM> of <FIG> to operate on a single machine, for example, the single host server <NUM>.

<FIG> is another illustration of virtualization in accordance with aspects of the present disclosure. In <FIG>, line replaceable units (LRUs) <NUM>, <NUM>, and <NUM> represent specialized hardware units, such as hardware units that can be used in an aircraft. Each of LRUs <NUM>, <NUM>, and <NUM> is simulated using one of VMs <NUM>, <NUM>, and <NUM>, respectively, on a single host server <NUM>. Although three LRUs (<NUM>, <NUM>, and <NUM>) are illustrated, it should be understood that a different number of LRUs can be combined as VMs on a host server. A virtual I/O component <NUM> manages the messaging and data flow among each of the VMs <NUM>, <NUM>, and <NUM> as guest machines. Operation in this manner permits the various components an aircraft to be virtualized to use with the VR environment platform <NUM> of <FIG> and the VR environment platform <NUM> of <FIG>.

An exemplary system provided herein is a VR environment platform. The VR environment platform comprises: an immersive VR environment engine comprising aircraft control and display units and providing an immersive VR environment of an aircraft; a virtual FMC running an OFP; and a communication channel coupling the immersive VR environment engine with the virtual FMC, the communication channel emulating FMC communication protocols to permit the immersive VR environment engine to run the virtual FMC.

An exemplary method provided herein is a method of operating an immersive VR environment. The method comprises: operating an immersive VR environment engine comprising aircraft control and display units and providing an immersive VR environment of an aircraft; operating a virtual FMC running an OFP; and coupling the immersive VR environment engine with the virtual FMC with a communication channel, the communication channel emulating FMC communication protocols to permit the immersive VR environment engine to run the virtual FMC.

An exemplary system provided herein includes one or more computer storage devices having computer-executable instructions stored thereon for operating an immersive VR environment platform, which, on execution by a computer, cause the computer to perform operations comprising: operating an immersive VR environment engine comprising aircraft control and display units and providing an immersive VR environment of an aircraft; displaying a view of an immersive VR environment on an HMD; receiving aircraft flight deck control inputs for the immersive VR environment through virtual interaction points; operating a virtual flight management computer (FMC) running an operational flight program (OFP); providing physical feedback (or receiving a physical input) to a user of the VR environment platform through a physical interaction point simulating an aircraft control; operating a flight simulator component interfacing with the immersive VR environment engine and the virtual FMC, and coupling the immersive VR environment engine with the virtual FMC with a protocol buffer, the communication channel emulating FMC communication protocols to permit the immersive VR environment engine to run the virtual FMC.

Alternatively or in addition to the other examples described herein, the foregoing exemplary system and methods include any combination of the following: an HMD for displaying a view of an immersive VR environment; displaying a view of an immersive VR environment on an HMD; virtual interaction points providing aircraft flight deck control inputs for the immersive VR environment; receiving aircraft flight deck control inputs for the immersive VR environment through virtual interaction points; a physical interaction point simulating an aircraft control and providing physical feedback to a user of the VR environment platform; providing physical feedback (or receiving a physical input) to a user of the VR environment platform through a physical interaction point simulating an aircraft control; a flight simulator component interfacing with the immersive VR environment engine and the virtual FMC; operating a flight simulator component interfacing with the immersive VR environment engine and the virtual FMC; an instructor operator station component interfacing with the flight simulator component; operating an instructor operator station component interfacing with the flight simulator component; a test environment core component interfacing with the flight simulator component, the immersive VR environment engine, and the virtual FMC; operating a test environment core component interfacing with the flight simulator component, the immersive VR environment engine, and the virtual FMC; at least one component interfacing with the test environment core component and selected from the list consisting of: navigation simulation, data link simulation, air traffic control simulation, and flight visualization; and interfacing with the test environment core component, at least one component selected from the list consisting of: navigation simulation, data link simulation, air traffic control simulation, and flight visualization.

The examples illustrated and described herein, as well as examples not specifically described herein but within the scope of aspects of the disclosure, constitute exemplary means for providing a VR environment. The order of execution or performance of the operations in examples of the disclosure illustrated and described herein is not essential, and can be performed in different sequential manners in various examples.

The terms "comprising," "including," and "having" are intended to be inclusive and mean that there can be additional elements other than the listed elements.

Claim 1:
A virtual reality, VR, environment platform (<NUM>) comprising:
an immersive VR environment engine (<NUM>) comprising aircraft control and display units (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>) and providing an immersive VR environment of an aircraft;
a virtual flight management computer, FMC, (<NUM>) implemented on a virtual machine and running an operational flight program, OFP, (<NUM>); and
a communication channel (<NUM>) coupling the immersive VR environment engine with the virtual FMC, the communication channel emulating FMC communication protocols to permit the immersive VR environment engine to run the virtual FMC.