Patent Description:
As medical science has progressed, it has become increasingly important to provide non-human interactive formats for teaching patient care. While it is desirable to train medical personnel in patient care protocols before allowing contact with real patients, textbooks and flash cards lack the important benefits to students that can be attained from hands-on practice. On the other hand, allowing inexperienced students to perform medical procedures on actual patients that would allow for the hands-on practice cannot be considered a viable alternative because of the inherent risk to the patient. Non-human interactive devices and systems can be used to teach the skills needed to successfully identify and treat various patient conditions without putting actual patients at risk.

For example, patient care education has often been taught using medical instruments to perform patient care activity on a physical simulator, such as a manikin. Such training devices and systems can be used by medical personnel and medical students to teach and assess competencies such as patient care, medical knowledge, practice based learning and improvement, systems based practice, professionalism, and communication. The training devices and systems can also be used by patients to learn the proper way to perform self-examinations. "In situ" simulation in healthcare is popular because it uses a real patient simulator in a real hospital environment. As a result, students are allowed to practice and make mistakes in the same area where they may later treat real life patients in a professional manner. However, one issue with this approach is that such facilities may be jammed with real patients and caregivers thus simulation time becomes limited.

The internal structure, functions, and processes of existing physical simulators are not visible by the user. In addition, at least some desirable external features are not present on the physical simulator or are poorly simulated by existing simulators. To address these issues, some physical simulators incorporate physically simulated fluids and physical disposables for a variety of treatment scenarios. However, physically simulated fluids have the potential of causing the physical simulator's electronics to short-circuit. Furthermore, physical disposables have a limited life-span, and are not amenable to anatomic variability on a large scale. Thus, while existing physical simulators have been adequate in many respects, they have not been adequate in all respects. Therefore, what is needed is an augmented reality system for use in conducting patient care training sessions that is even more realistic and/or includes additional simulated features.

<CIT> discloses an interactive mixed reality simulator is provided that includes a virtual 3D model of internal or hidden features of an object; a physical model or object being interacted with; and a tracked instrument used to interact with the physical object. The tracked instrument can be used to simulate or visualize interactions with internal features of the physical object represented by the physical model. In certain embodiments, one or more of the internal features can be present in the physical model. In another embodiment, some internal features do not have a physical presence within the physical model.

<CIT> discloses an interactive education system for teaching patient care to a user is described. The system comprises a patient simulator; a virtual instrument for use with the patient simulator in performing patient care activities; means for sensing an interaction between the virtual instrument and the simulator, and means for providing feedback to the user regarding the interaction between the virtual instrument and the simulator. In one aspect, the system includes a maternal simulator, a fetal simulator, and a neonatal simulator.

It will nevertheless be understood that no limitation of the scope of the disclosure is intended. Any alterations and further modifications to the described devices, instruments, methods, and any further application of the principles of the present disclosure are fully contemplated as would normally occur to one skilled in the art to which the disclosure relates. For simplicity, in some instances the same reference numbers are used throughout the drawings to refer to the same or like parts.

One of the aims of healthcare simulation is to establish a teaching environment that closely mimics key clinical cases in a reproducible manner. The introduction of high fidelity tetherless simulators, such as those available from Gaumard Scientific Company, Inc. , over the past few years has proven to be a significant advance in creating realistic teaching environments. The present disclosure is directed to an augmented reality ("AR") system for teaching patient care that expands the functionality of the simulators by increasing the realism of the look, feel, and functionality of the simulators that can be used to train medical personnel in a variety of clinical situations. The AR system disclosed herein offers a training platform on which team-building scenarios can be performed for the development of medical treatment skills and the advancement of patient safety.

In particular, the AR system disclosed herein may include, or be part of, a patient simulator to provide improved realism and functionality compared to previously available simulators. Some of the various features that facilitate the improved realism and functionality are described in detail below. The AR system of the present disclosure allows users to practice a range of different scenarios.

Thus, the AR system facilitates the training of a user across a broad range of simulated scenarios and corresponding assessment of the user's response to the different simulated scenarios. Accordingly, the user's medical treatment skills can be obtained and/or improved in a simulated environment without endangering a live patient.

Moreover, the AR system allows for multiple users to simultaneously work with the patient simulator during a particular birthing and/or neonatal scenario, thereby facilitating team training and assessment in a realistic, team-based environment. By allowing multiple users to simultaneously interact with the AR system, the system facilitates the real-time training and assessment of the cooperative efforts of a team in a wide variety of scenarios, such as, by way of non-limiting example, a fire in the hospital. In some embodiments, the AR system provides for pre-operative care simulation as well as post-operative care simulation, thereby allowing users to experience, address, and assess pre-operative and post-operative management, including pre-operative acquisition of the patient history and management of post-operative complications.

For example, in some embodiments, the AR system allows for the realistic reception and transport of the patient simulator through a hospital (e.g., from an emergency room to an operating room) during operation of a particular scenario. In addition, the AR system can be used to conduct patient safety drills in an actual hospital or other medical setting.

In some embodiments, the AR system includes features designed to enhance the educational experience. For example, in some embodiments, the system includes a processing module to simulate different medical and/or surgical scenarios during operation of the AR system. In some embodiments, the system includes a camera system that allows visualization of the procedure for real-time video and log capture for debriefing purposes. In some embodiments, the AR system is provided with a workbook of medical scenarios that are pre-programmed in an interactive software package, thereby providing a platform on which team-building scenarios can be performed for the development of medical treatment skills and general patient safety. Thus, the AR system disclosed herein provides a system that is readily expandable and updatable without large expense and that enables users to learn comprehensive medical and surgical skills through "hands-on" training, without sacrificing the experience gained by users in using standard surgical instruments in a simulated patient treatment situation.

The present disclosure introduces AR applications wherein a virtual world, static or dynamic, is superimposed onto a real physical simulator so that when the student(s) have AR headset devices, they will see both the real simulator and the virtual overlay in a manner which will improve the rate of learning. Going a step further, augmented reality can be viewed as a series of overlays. For example, a basic environment (which could be a hospital ER for example) is recorded on an AR headset device (such as Hololens® from Microsoft®). Avatars may then be placed into this base environment, including people such as nurses, doctors, and significant others who may move and speak during a particular scenario. Moreover, a physical or virtual simulator may be placed in this base environment. As a result, student(s) will feel as though they have also been placed within the base environment together with with the physical or virtual simulator, and may treat the physical or virtual simulator as appropriate. The student(s)' choice of activities and their results may then be recorded to memorialize clinical competency.

Referring initially to <FIG>, an AR system for teaching patient care is generally referred to by the reference numeral <NUM>. The AR system <NUM> includes a physical anatomic model <NUM>, a computing device <NUM>, a display unit <NUM>, and a tracking system <NUM>. The computing device <NUM>, the display unit <NUM>, and the tracking system <NUM> are connected to a network <NUM> (e.g., LAN or WAN). In addition, or instead, the computing device <NUM>, the display unit <NUM>, and the tracking system <NUM> may be interconnected via another wired or wireless link. As shown in <FIG>, a user <NUM> receives optic feedback from the physical anatomic model <NUM> through the display unit <NUM>, as indicated by the arrow <NUM>. In addition, the user <NUM> receives haptic feedback from the physical anatomic model <NUM> via direct physical contact, as indicated by the arrow <NUM>. In some embodiments, the AR system <NUM> further includes an instrument <NUM> via which the user <NUM> physically interacts with the physical anatomic model <NUM>, such as, for example, motion-tracked gloves, an IV needle, an endotracheal (ET) tube, an electrocardiogram (ECG or EKG) monitor, a blood pressure (BP) cuff, a pulse oximeter cuff, a temporary external pacer, an automatic external defibrillator (AED), a manual defibrillator, an ultrasound wand, a stethoscope, a thermometer, a fetal distress monitor, another diagnostic or surgical instrument, or any combination thereof.

The display unit <NUM> is wearable by the user <NUM>, and is thus also referred to herein as an AR headset device <NUM>. In addition, or instead, the display unit <NUM> may be handheld or mounted in a stationary position. Accordingly, each embodiment described herein as including the AR headset device <NUM> is equally operable with another suitable display unit, such as a handheld display unit or a display unit mounted in a stationary position. In some embodiments, to permit the user <NUM>'s receipt of the optic feedback <NUM> from the physical anatomic model <NUM> via the AR headset device <NUM>, the AR headset device <NUM> includes a transparent (or semi-transparent) lens (not shown). In some embodiments, to permit the user <NUM>'s receipt of the optic feedback <NUM> from the physical anatomic model <NUM> via the AR headset device <NUM>, the AR headset device <NUM> includes a screen (not shown) and an integrated camera (not shown) that captures footage of the physical anatomic model <NUM> to display on the screen in real-time. In some embodiments, the AR headset device <NUM> includes, or is part of, at least a portion of the tracking system <NUM> and/or the computing device <NUM>. Alternatively, the AR headset device <NUM> may include an onboard computing device separate from, but substantially similar to, the computing device <NUM> to run an AR application locally on the AR headset device <NUM>, as will be described in further detail below.

The tracking system <NUM> tracks the position and orientation of the AR headset device <NUM> in three-dimensional space and relative to the physical anatomic model <NUM>. In some embodiments, the tracking system <NUM> tracks the position and orientation of the AR headset device <NUM> with six degrees-of-freedom ("<NUM>-DoF"), including x, y, and z coordinates of the AR headset device <NUM>, and pitch, yaw, and roll of the AR headset device <NUM>. The tracking system <NUM> may be any suitable type of tracking system capable of tracking the position and orientation of the AR headset device <NUM> (e.g., tracking fiducial markers, using stereo images to track retro-reflective infrared markers, employing electromagnetic tracker(s), etc.). In some embodiments, at least a portion of the tracking system <NUM> includes, or is part of, the AR headset device <NUM> and/or the computing device <NUM>. The tracking system <NUM> can include sensors embedded in the AR headset device <NUM>, including without limitation gyroscope(s), accelerometer(s), GPS sensor(s), and/or combinations thereof. A holographic rendering of the physical world (e.g., a 3D mesh model of the physical world) can be utilized to coordinate the virtual positioning to the physical world. For example, a holographic computer and head-mounted display, such as the Hololens® available from Microsoft®, may be used to provide a holographic unit to render virtual objects in the physical world.

The computing device <NUM> is capable of receiving, via the network <NUM>, signals from the tracking system <NUM> relating to the position and orientation of the AR headset device <NUM>. Moreover, based on the signals received from the tracking system <NUM>, the computing device <NUM> is capable of sending, via the network <NUM>, appropriate signals to the AR headset device <NUM> to augment or otherwise enhance the user <NUM>'s view of the physical anatomic model <NUM>, as will be discussed in further detail below. In some embodiments, the computing device <NUM> includes, or is part of, the AR headset device <NUM> and/or at least a portion of the tracking system <NUM>.

Turning to <FIG>, the physical anatomic model <NUM> includes a manikin <NUM>. The manikin <NUM> includes external physical features <NUM>. The external physical features <NUM> of the manikin <NUM> may include, for example, physical representations of one or more external characteristics associated with a natural human's torso, legs, arms, head, or any combination thereof; such external physical features <NUM> of the manikin <NUM> provide both optic feedback (as indicated by the arrow 24aa in <FIG>) and haptic feedback (as indicated by the arrow 26a in <FIG>) to the user <NUM>. More particularly, the optic feedback 24aa received by the user <NUM> emanates from the manikin <NUM> and passes through the AR headset device <NUM>. In some embodiments, the manikin <NUM> also includes internal physical structures <NUM>. The internal physical structures <NUM> of the manikin <NUM> may include physical representations of one or more internal characteristics associated with the natural human's torso, such as, for example, the spine, the ribs, the heart, the lungs, the liver, another internal characteristic of the natural human's torso, or any combination thereof. In addition, or instead, the internal physical structures <NUM> of the manikin <NUM> may include, for example, physical representations of one or more internal characteristics associated with the natural human's legs, arms, head, or any combination thereof. The internal physical structures <NUM> of the manikin <NUM> provide haptic feedback (as indicated by the arrow 26b in <FIG>), and not optic feedback, to the user <NUM>. The internal physical structures <NUM> can include female reproductive system, cardiovascular system associated with venous/arterial access, central nervous system associated with lumbar puncture, major organs and distinctive representations based on simulated scenario (e.g., healthy lungs vs. lungs affected with COPD, organs squashed during pregnancy, removing a gall stone endoscopically, etc.). The external physical structures <NUM> can include scars, skin anomalies, distinctive skin marks, skin discoloration, cyanosis, wounds, etc..

To augment or otherwise enhance the user <NUM>'s view of the physical anatomic model <NUM>, a virtual anatomic model <NUM> is overlaid on the user <NUM>'s view of the physical anatomic model <NUM> via the AR headset device <NUM>. More particularly, the virtual anatomic model <NUM> is displayed on the AR headset device <NUM> within the user <NUM>'s field of view so that the user <NUM> simultaneously views both the physical anatomic model <NUM> and the virtual anatomic model <NUM>. The virtual anatomic model <NUM> is stored on, or accessible by, the computing device <NUM>. In addition to the virtual anatomic model <NUM>, a plurality of virtual anatomic models (not shown) may be stored on, or accessible by, the computing device <NUM> to simulate a wide variety of anatomies and pathologies encountered during the particular procedure being trained for. In some embodiments, the physical anatomic model <NUM> and the virtual anatomic model <NUM>, in combination, represent characteristics of the natural human. The virtual anatomic model <NUM> includes virtual anatomy <NUM>.

The virtual anatomy <NUM> includes internal virtual structures <NUM>. The internal virtual features <NUM> of the virtual anatomy <NUM> may include virtual representations of one or more internal characteristics associated with the natural human's torso, such as, for example, the spine, the ribs, the heart, the lungs, the liver, another internal characteristic of the natural human's torso, or any combination thereof. In addition, or instead, the internal virtual features <NUM> of the virtual anatomy <NUM> may include, for example, virtual representations of one or more internal characteristics associated with the natural human's legs, arms, head, or any combination thereof. The internal virtual features <NUM> of the virtual anatomy <NUM> provide optic feedback (as indicated by the arrow 24b in <FIG>), and not haptic feedback, to the user <NUM>. The internal virtual features <NUM> can include aspects of the female reproductive system, cardiovascular system associated with venous/arterial access, central nervous system associated with lumbar puncture, major organs and distinctive representations based on simulated scenario (e.g., healthy lungs vs. lungs affected with COPD, organs squashed during pregnancy, removing a gall stone endoscopically, etc.).

In some embodiments, the virtual anatomy <NUM> also includes external virtual features <NUM> that provide enhanced photorealism to the user <NUM>'s view of the manikin <NUM>'s external physical features <NUM>. The external virtual features <NUM> of the virtual anatomy <NUM> may include, for example, virtual representations of one or more external characteristics associated with the natural human's torso, legs, arms, head, or any combination thereof; such external virtual features <NUM> of the virtual anatomy <NUM> provide optic feedback (as indicated by the arrow 24ab in <FIG>), and not haptic feedback, to the user <NUM>. In some embodiments, the virtual anatomy <NUM> is co-registered (using the computing device <NUM>) with the manikin <NUM> so that the internal virtual structures <NUM> and the external virtual features <NUM> of the virtual anatomy <NUM> have an accurate spatial relationship with the internal physical structures <NUM> and the external physical features <NUM>, respectively, of the manikin <NUM>. The external virtual features <NUM> can include aspects of scars, skin anomalies, distinctive skin marks, skin discoloration, cyanosis, wounds, etc..

The external physical features <NUM> of the manikin <NUM> and the external virtual features <NUM> of the virtual anatomy <NUM> are configurable to realistically simulate the external characteristics associated with the natural human by providing the user <NUM> with an appropriate combination of optic and haptic feedback (as indicated by the arrows 24aa, 24ab, and 26a, respectively, in <FIG>). To this end, the external physical features <NUM> of the manikin <NUM> may simulate some external characteristics associated with the natural human while other external characteristics associated with the natural human are simulated by the external virtual features <NUM> of the virtual anatomy <NUM>; in addition, or instead, one or more external characteristics associated with the natural human may be simulated by both the external physical features <NUM> of the manikin <NUM> and the external virtual features <NUM> of the virtual anatomy <NUM>.

The internal physical structures <NUM> of the manikin <NUM> and the internal virtual structures <NUM> of the virtual anatomy <NUM> are configurable to realistically simulate the internal characteristics associated with the natural human by providing the user <NUM> with an appropriate combination of optic and haptic feedback (as indicated by the arrows 24b and 26b, respectively, in <FIG>). To this end, the internal physical structures <NUM> of the manikin <NUM> may simulate some internal characteristics associated with the natural human while other internal characteristics associated with the natural human are simulated by the internal virtual structures <NUM> of the virtual anatomy <NUM>; in addition, or instead, one or more internal characteristics associated with the natural human may be simulated by both the internal physical structures <NUM> of the manikin <NUM> and the internal virtual structures <NUM> of the virtual anatomy <NUM>.

In operation, the virtual anatomic model <NUM> and the physical anatomic model <NUM> illustrated in <FIG>, in combination, provide a training platform for simulating, inter alia, proper tracheostomy procedures (including insertion of a trachea tube), proper pneumothorax procedures, wound hemorrhaging (including proper application of packing pressure as well as, alternatively, proper implementation of an adequate tourniquet at a site suitable to stop the wound(s) from further blood loss), proper treatment of lesions or lacerations caused by battle, combat, explosion, trauma, other diagnostic, treatment, or surgical scenarios, including birthing and obstetrics procedures, or any combination thereof. During these simulations, the external physical features <NUM> of the manikin <NUM> are visible to the user <NUM>, but the internal physical structures <NUM> of the manikin <NUM> are not visible to the user <NUM>. The tracking system <NUM> tracks the position and orientation of the AR headset device <NUM> and the computing device <NUM> receives, via the network <NUM>, signals from the tracking system <NUM> relating to the position and orientation of the AR headset device <NUM>. The computing device <NUM> sends, via the network <NUM>, appropriate signals to the AR headset device <NUM> to overlay the virtual anatomic model <NUM> (including the virtual anatomy <NUM>) on the physical anatomic model <NUM>. As a result, the AR headset device <NUM> accurately overlays the virtual anatomy <NUM> on the user <NUM>'s view of the manikin <NUM>.

Turning to <FIG>, the physical anatomic model <NUM> is omitted and replaced with a physical anatomic model <NUM>. The physical anatomic model <NUM> includes a maternal manikin <NUM>. The maternal manikin <NUM> includes external physical features <NUM>. The external physical features <NUM> of the maternal manikin <NUM> may include, for example, physical representations of one or more external characteristics associated with a natural mother's torso, legs, arms, head, or any combination thereof; such external physical features <NUM> of the maternal manikin <NUM> provide both optic feedback (as indicated by the arrow 24ca in <FIG>) and haptic feedback (as indicated by the arrow 26c in <FIG>) to the user <NUM>. More particularly, the optic feedback 24ca received by the user <NUM> emanates from the maternal manikin <NUM> and passes through the AR headset device <NUM>. In some embodiments, the maternal manikin <NUM> also includes internal physical structures <NUM>. The internal physical structures <NUM> of the maternal manikin <NUM> may include physical representations of one or more internal characteristics associated with the natural mother's torso, such as, for example, the spine, the ribs, the pubic bone, the uterus, the cervix, another internal characteristic of the natural mother's torso, or any combination thereof. In addition, or instead, the internal physical structures <NUM> of the maternal manikin <NUM> may include, for example, physical representations of one or more internal characteristics associated with the natural mother's legs, arms, head, or any combination thereof. The internal physical structures <NUM> of the maternal manikin <NUM> provide haptic feedback (as indicated by the arrow 26d in <FIG>), and not optic feedback, to the user <NUM>. The internal physical structures <NUM> can include female reproductive system, cardiovascular system associated with venous/arterial access, central nervous system associated with lumbar puncture, major organs and distinctive representations based on simulated scenario (e.g., healthy lungs vs. lungs affected with COPD, organs squashed during pregnancy, removing a gall stone endoscopically, etc.). The external physical features <NUM> can include scars, skin anomalies, distinctive skin marks, skin discoloration, cyanosis, wounds, etc..

To augment or otherwise enhance the user <NUM>'s view of the physical anatomic model <NUM>, a virtual anatomic model <NUM> is overlaid on the user <NUM>'s view of the physical anatomic model <NUM> using the AR headset device <NUM>. More particularly, the virtual anatomic model <NUM> is displayed on the AR headset device <NUM> within the user <NUM>'s field of view so that the user <NUM> simultaneously views both the physical anatomic model <NUM> and the virtual anatomic model <NUM>. The virtual anatomic model <NUM> is stored on, or accessible by, the computing device <NUM>. In addition to the virtual anatomic model <NUM>, a plurality of virtual anatomic models (not shown) may be stored on, or accessible by, the computing device <NUM> to simulate a wide variety of anatomies and pathologies encountered during the particular procedure being trained for. In some embodiments, the physical anatomic model <NUM> and the virtual anatomic model <NUM>, in combination, represent characteristics of, and interactions between, the natural mother and a natural fetus. The virtual anatomic model <NUM> includes virtual maternal anatomy <NUM> and virtual fetal anatomy <NUM>.

The virtual maternal anatomy <NUM> includes internal virtual structures <NUM>. The internal virtual structures <NUM> of the virtual maternal anatomy <NUM> may include virtual representations of one or more internal characteristics associated with the natural mother's torso, such as, for example, the spine, the ribs, the pubic bone, the uterus, the cervix, another internal characteristic of the natural mother's torso, or any combination thereof. In addition, or instead, the internal virtual structures <NUM> of the virtual maternal anatomy <NUM> may include, for example, virtual representations of one or more internal characteristics associated with the natural mother's legs, arms, head, or any combination thereof. The internal virtual structures <NUM> of the virtual maternal anatomy <NUM> provide optic feedback (as indicated by the arrow 24d in <FIG>), and not haptic feedback, to the user <NUM>. The internal virtual structures <NUM> can include aspects of the female reproductive system, cardiovascular system associated with venous/arterial access, central nervous system associated with lumbar puncture, major organs and distinctive representations based on simulated scenario (e.g., healthy lungs vs. lungs affected with COPD, organs squashed during pregnancy, removing a gall stone endoscopically, etc.).

In some embodiments, the virtual maternal anatomy <NUM> also includes external virtual features <NUM> that provide enhanced photorealism to the user <NUM>'s view of the maternal manikin <NUM>'s external physical features <NUM>. The external virtual features <NUM> of the virtual maternal anatomy <NUM> may include, for example, virtual representations of one or more external characteristics associated with the natural mother's torso, legs, arms, head, or any combination thereof; such external virtual features <NUM> of the virtual maternal anatomy <NUM> provide optic feedback (as indicated by the arrow 24cb in <FIG>), and not haptic feedback, to the user <NUM>. The external virtual structures <NUM> can include scars, skin anomalies, distinctive skin marks, skin discoloration, cyanosis, wounds, etc..

The external physical features <NUM> of the maternal manikin <NUM> and the external virtual features <NUM> of the virtual maternal anatomy <NUM> are configurable to realistically simulate the external characteristics associated with the natural mother by providing the user <NUM> with an appropriate combination of optic and haptic feedback (as indicated by the arrows 24ca, 24cb, and 26c, respectively, in <FIG>). To this end, the external physical features <NUM> of the maternal manikin <NUM> may simulate some external characteristics associated with the natural mother while other external characteristics associated with the natural mother are simulated by the external virtual features <NUM> of the virtual maternal anatomy <NUM>; in addition, or instead, one or more external characteristics associated with the natural mother may be simulated by both the external physical features <NUM> of the maternal manikin <NUM> and the external virtual features <NUM> of the virtual maternal anatomy <NUM>.

The internal physical structures <NUM> of the maternal manikin <NUM> and the internal virtual structures <NUM> of the virtual maternal anatomy <NUM> are configurable to realistically simulate the internal characteristics associated with the natural mother by providing the user <NUM> with an appropriate combination of optic and haptic feedback (as indicated by the arrows 24d and 26d, respectively, in <FIG>). To this end, the internal physical structures <NUM> of the maternal manikin <NUM> may simulate some internal characteristics associated with the natural mother while other internal characteristics associated with the natural mother are simulated by the internal virtual structures <NUM> of the virtual maternal anatomy <NUM>; in addition, or instead, one or more internal characteristics associated with the natural mother may be simulated by both the internal physical structures <NUM> of the maternal manikin <NUM> and the internal virtual structures <NUM> of the virtual maternal anatomy <NUM>.

The virtual fetal anatomy <NUM> includes external virtual features <NUM>. The external virtual features <NUM> of the virtual fetal anatomy <NUM> may include, for example, virtual representations of one or more external characteristics associated with the natural fetus' torso, legs, arms, head, or any combination thereof. In addition, the external virtual features <NUM> of the virtual fetal anatomy <NUM> may include, for example, virtual representations of the natural fetus' amniotic sac, placenta, umbilical cord, or any combination thereof. The external virtual features <NUM> of the virtual fetal anatomy <NUM> provide optic feedback (as indicated by the arrow 24e in <FIG>), and not haptic feedback, to the user <NUM>. In some embodiments, the virtual fetal anatomy <NUM> also includes internal virtual structures <NUM> to enhance training for, e.g., intrauterine fetal procedure(s). The internal virtual structures <NUM> of the virtual fetal anatomy <NUM> may include, for example, virtual representations of internal characteristics associated with the natural fetus' torso, legs, arm, head, or any combination thereof; such internal virtual structures <NUM> of the virtual fetal anatomy <NUM> provide optic feedback (as indicated by the arrow 24f in <FIG>), and not haptic feedback, to the user <NUM>.

The AR system <NUM>'s physical anatomic model <NUM> further includes a fetal manikin <NUM> contained within the maternal manikin <NUM>. The fetal manikin <NUM> includes external physical features <NUM>. The external physical features <NUM> of the fetal manikin <NUM> may include, for example, physical representations of one or more external characteristics associated with the natural fetus' torso, legs, arms, head, or any combination thereof. In addition, the external physical features <NUM> of the fetal manikin <NUM> may include, for example, physical representations of the natural fetus' amniotic sac, placenta, umbilical cord, or any combination thereof. As a result of the fetal manikin <NUM>'s containment within the maternal manikin <NUM>, the external physical features <NUM> of the fetal manikin <NUM> provide haptic feedback (as indicated by the arrow 26e in <FIG>), and not optic feedback, to the user <NUM>. In some embodiments, the fetal manikin <NUM> also includes internal physical structures <NUM> to enhance training for, e.g., intrauterine fetal procedure(s). The internal physical structures <NUM> of the fetal manikin <NUM> may include, for example, physical representations of one or more internal characteristics associated with the natural fetus' torso, legs, arms, head, or any combination thereof; such internal physical structures <NUM> of the fetal manikin <NUM> provide haptic feedback (as indicated by the arrow 26f in <FIG>), and not optic feedback, to the user <NUM>.

The external physical features <NUM> of the fetal manikin <NUM> and the external virtual features <NUM> of the virtual fetal anatomy <NUM> are configurable to realistically simulate the external characteristics associated with the natural fetus by providing the user <NUM> with an appropriate combination of optic and haptic feedback (as indicated by the arrows 24e and 26e, respectively, in <FIG>). To this end, the external physical features <NUM> of the fetal manikin <NUM> may simulate some external characteristics associated with the natural fetus while other external characteristics associated with the natural fetus are simulated by the external virtual features <NUM> of the virtual fetal anatomy <NUM>; in addition, or instead, one or more external characteristics associated with the natural fetus may be simulated by both the external physical features <NUM> of the fetal manikin <NUM> and the external virtual features <NUM> of the virtual fetal anatomy <NUM>.

The internal physical structures <NUM> of the fetal manikin <NUM> and the internal virtual structures <NUM> of the virtual fetal anatomy <NUM> are configurable to realistically simulate the internal characteristics associated with the natural fetus by providing the user <NUM> with an appropriate combination of optic and haptic feedback (as indicated by the arrows 24f and 26f, respectively, in <FIG>). To this end, the internal physical structures <NUM> of the fetal manikin <NUM> may simulate some internal characteristics associated with the natural fetus while other internal characteristics associated with the natural fetus are simulated by the internal virtual structures <NUM> of the virtual fetal anatomy <NUM>; in addition, or instead, one or more internal characteristics associated with the natural fetus may be simulated by both the internal physical structures <NUM> of the fetal manikin <NUM> and the internal virtual structures <NUM> of the virtual fetal anatomy <NUM>.

In addition to tracking the position and orientation of the AR headset device <NUM>, the tracking system <NUM> is capable of tracking the position and orientation of the fetal manikin <NUM> in three-dimensional space and relative to the maternal manikin <NUM>. In some embodiments, the tracking system <NUM> tracks the position and orientation of the fetal manikin <NUM> with six degrees-of-freedom ("<NUM>-DoF"), including x, y, and z coordinates of the fetal manikin <NUM>, and pitch, yaw, and roll of the fetal manikin <NUM>. The tracking system <NUM> may be any suitable type of tracking system capable of tracking the position and orientation of the fetal manikin <NUM> (e.g., tracking fiducial markers, using stereo images to track retro-reflective infrared markers, employing electromagnetic tracker(s), etc.). In some embodiments, at least a portion of the tracking system <NUM> includes, or is part of, the fetal manikin <NUM>. Accordingly, in addition to receiving, via the network <NUM>, signals from the tracking system <NUM> relating to the position and orientation of the AR headset device <NUM>, the computing device <NUM> is capable of receiving, via the network <NUM>, signals from the tracking system <NUM> relating to the position and orientation of the fetal manikin <NUM>. In some instances, the tracking system tracks the position and orientation of the fetal manikin <NUM> relative to the maternal manikin <NUM> using one or more trackable markers (e.g., physical, infrared, RFID, electromagnetic, etc.) placed on (or in) the fetal manikin and computer vision (e.g., using a suitable camera or other tracking mechanism for the type or marker(s)). In some instances, the tracking system tracks the position and orientation of the fetal manikin <NUM> relative to the maternal manikin <NUM> using feedback from the maternal manikin <NUM> regarding a position of a birthing mechanism of the maternal manikin <NUM> that defines the position and orientation of the fetal manikin.

In some embodiments, the virtual maternal anatomy <NUM> is co-registered (using the computing device <NUM>) with the maternal manikin <NUM> so that the internal virtual structures <NUM> and the external virtual features <NUM> of the virtual maternal anatomy <NUM> have an accurate spatial relationship with the internal physical structures <NUM> and the external physical features <NUM>, respectively, of the maternal manikin <NUM>. Similarly, in some embodiments, the virtual fetal anatomy <NUM> is co-registered (using the computing device <NUM>) with the maternal manikin so that the external virtual features <NUM> and the internal virtual structures <NUM> of the virtual fetal anatomy <NUM> have an accurate spatial relationship with the maternal manikin <NUM>. In addition, the virtual fetal anatomy <NUM> is co-registered (using the computing device <NUM>) with the fetal manikin <NUM> so that the external virtual features <NUM> and the internal virtual structures <NUM> of the virtual fetal anatomy <NUM> have an accurate spatial relationship with the external physical features <NUM> and the internal physical structures <NUM>, respectively, of the fetal manikin <NUM>. In some instances, the co-registration is accomplished by saving a spatial mapping of the physical environment and assigning anchors to the real world. Further, in some instances, user defined placement of the virtual anatomy on the physical manikin can be used, alone or with the spatial mapping, to facilitate the co-registration.

In operation, the virtual anatomic model <NUM> and the physical anatomic model <NUM> illustrated in <FIG>, in combination, provide a training platform for simulating, inter alia, routine gestational palpation of the natural fetus, Leopold's Maneuvers to determine the position of the natural fetus inside the natural mother's uterus, an external cephalic version to turn the natural fetus from a breech position or side-lying (transverse) position into a head-down (vertex) position before labor begins, one or more intrauterine fetal procedure(s), true-to-life shoulder dystocia, breech, and C-section deliveries, other diagnostic, treatment, or surgical scenarios, including birthing and obstetrics procedures, or any combination thereof. During these simulations, the external physical features <NUM> of the maternal manikin <NUM> are visible to the user <NUM>, but the fetal manikin <NUM> and the internal physical structures <NUM> of the maternal manikin <NUM> are not visible to the user <NUM>. The tracking system <NUM> tracks the position and orientation of the AR headset device <NUM> and the computing device <NUM> receives, via the network <NUM>, signals from the tracking system <NUM> relating to the position and orientation of the AR headset device <NUM>. In addition, the tracking system <NUM> tracks the position and orientation of the fetal manikin <NUM> relative to the maternal manikin <NUM>, and the computing device <NUM> receives, via the network <NUM>, signals from the tracking system <NUM> relating to the position and orientation of the fetal manikin <NUM>. The computing device <NUM> sends, via the network <NUM>, appropriate signals to the AR headset device <NUM> to overlay the virtual anatomic model <NUM> (including the virtual maternal anatomy <NUM> and the virtual fetal anatomy <NUM>) on the physical anatomic model <NUM>. More particularly, because the virtual maternal anatomy <NUM> is co-registered (using the computing device <NUM>) with the maternal manikin <NUM>, and the virtual fetal anatomy <NUM> is co-registered (using the computing device <NUM>) with the fetal manikin <NUM>, any change in the position and orientation of the fetal manikin <NUM> relative to the maternal manikin <NUM> is reflected in the AR headset device <NUM>'s overlaying of the virtual anatomic model <NUM> on the physical anatomic model <NUM> via the AR headset device <NUM>. As a result, the virtual maternal anatomy <NUM> and the virtual fetal anatomy <NUM> are accurately overlaid on the user <NUM>'s view of the maternal manikin <NUM> via the AR headset device <NUM>.

Turning back to <FIG>, with continuing reference to <FIG> and <FIG>, in some embodiments, the user <NUM> physically interacts with the physical anatomic model <NUM> or the physical anatomic model <NUM> using the instrument <NUM>. In some embodiments, in addition to tracking the position and orientation of the AR headset device <NUM>, the tracking system <NUM> is capable of tracking the position and orientation of the instrument <NUM> in three-dimensional space. In some embodiments, the tracking system <NUM> tracks the position and orientation of the instrument <NUM> with six degrees-of-freedom ("<NUM>-DoF"), including x, y, and z coordinates of the instrument <NUM>, and pitch, yaw, and roll of the instrument <NUM>. In some embodiments, at least a portion of the tracking system <NUM> includes, or is part of, the instrument <NUM>. The computing device <NUM> is capable of receiving, via the network <NUM>, signals from the tracking system <NUM> relating to the position and orientation of the instrument <NUM>. As a result, the tracking system <NUM> may be used to determine whether the instrument <NUM> is impinging on the virtual anatomic model <NUM> or the virtual anatomic model <NUM>. More particularly, based on the position and orientation of the instrument <NUM>, deformable portions of the virtual anatomic model <NUM> or the virtual anatomic model <NUM> are made to appropriately reflect, via the AR headset device <NUM>, the distortion caused by the impinging instrument <NUM>. For example, a blood vessel may become less circular and flatter when pressure is applied to it via the instrument <NUM>. In some instances, the tracking system tracks the position and orientation of the instrument <NUM> using one or more trackable markers (e.g., physical, infrared, RFID, electromagnetic, etc.) placed on (or in) the instrument and computer vision (e.g., using a suitable camera or other tracking mechanism for the type or marker(s)) of the instrument <NUM>.

Referring to <FIG> and <FIG>, to further augment or otherwise enhance the training experience, ancillary virtual graphics <NUM> are presented in the user <NUM>'s field of view via the AR headset device <NUM>. The ancillary virtual graphics <NUM> include, but are not limited to, medical data <NUM>, instructional steps <NUM>, expert demonstrations <NUM>, didactic content <NUM>, and exigent circumstances <NUM>. The medical data <NUM>, the instructional steps <NUM>, the expert demonstrations <NUM>, the didactic content <NUM>, and the exigent circumstances <NUM> provide optic feedback (as indicated by the arrows <NUM>-k, respectively, in <FIG> and <FIG>), and not haptic feedback, to the user <NUM>. The medical data <NUM> may include, but is not limited to, temperature, blood pressure, pulse, respiration rate, other critical medical data, or any combination thereof. The instructional steps <NUM> may include, but are not limited to, steps for completing the particular procedure being trained. Thus, in the embodiment of <FIG> and <FIG>, the instructional steps <NUM> may include steps for completing, for example, routine gestational palpation of the natural fetus, Leopold's Maneuvers to determine the position of the natural fetus inside the natural mother's uterus, an external cephalic version to turn the natural fetus from a breech position or side-lying (transverse) position into a head-down (vertex) position before labor begins, one or more intrauterine fetal procedure(s), true-to-life shoulder dystocia, breech, and C-section deliveries, other diagnostic, treatment, or surgical scenarios, or any combination thereof. In addition, or instead, in the embodiment of <FIG> and <FIG>, the instructional steps may include steps <NUM> for completing, for example, a proper tracheostomy procedure (including insertion of a trachea tube), a proper pneumothorax procedure, wound hemorrhaging (including proper application of packing pressure as well as, alternatively, proper implementation of an adequate tourniquet at a site suitable to stop the wound(s) from further blood loss), proper treatment of lesions or lacerations caused by battle, combat, explosion, or trauma, other diagnostic, treatment, or surgical scenarios, or any combination thereof. The expert demonstrations <NUM> supplement the instructional steps <NUM> and may be overlaid on the physical anatomic model <NUM> via the AR headset device <NUM> to demonstrate the correct way to complete one or more of the instructional steps <NUM>. The didactic content <NUM> provides educational materials such as, for example, medical research, scientific data, etc., to further enrich the training experience. Finally, the exigent circumstances <NUM> are overlaid on, or peripheral to, the user <NUM>'s view of the physical anatomic model <NUM> via the AR headset device <NUM>. The exigent circumstances <NUM> may include, but are not limited to, physical trauma (e.g., hemorrhaging), delivery room conditions, operating room conditions, fire, battlefield conditions, one or more other exigencies related to the procedure being trained for, or any combination thereof. In addition to self-guided teaching data, such as cardiopulmonary resuscitation feedback from the physical manikin (e.g., compression rate and depth), the AR headset device <NUM> can show medical data <NUM> (e.g., vital signs and/or other measured medical data) in real time.

While example functionalities of the physical anatomic models <NUM> and <NUM> are described above, no limitation is intended thereby. Rather, it is understood that the concepts of the present disclosure are applicable to a wide range of medical simulation functionalities and features. Accordingly, in some instances, the physical anatomic models <NUM> and <NUM> each include one or more features as described in the context of the simulators disclosed in: <CIT>; <CIT>, now <CIT>; <CIT>, now <CIT>; <CIT>, published as <CIT>; <CIT>, now <CIT>; <CIT>, now <CIT>; <CIT>, now <CIT>; <CIT>, now <CIT>; <CIT>, now <CIT>; <CIT>, now <CIT>; <CIT>, now <CIT>; <CIT>, now <CIT>; <CIT>, published as <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; <CIT>; and <CIT>.

Further, in some instances, the physical anatomic models <NUM> and <NUM> each include one or more features as provided in medical simulators provided by Gaumard Scientific Company, Inc. based out of Miami, Fla. , including but not limited to the following models: S1000 Hal®, S1020 Hal®, S1030 Hal®, S3000 Hal®, S2000 Susie®, S221 Clinical Chloe®, S222 Clinical Chloe®, S222. <NUM> Super Chloe®, S303 Code Blue®, S304 Code Blue®, S100 Susie®, S100 Simon®, S200 Susie®, S200 Simon®, S201 Susie®, S201 Simon®, S203 Susie®, S204 Simon®, S205 Simple Simon®, S206 Simple Susie®, S3004 Pediatric Hal®, S3005 Pediatric Hal®, S3009 Premie Hal®, S3010 Newborn Hal®, S110 Mike®, S110 Michelle®, S150 Mike®, S150 Michelle®, S107 Multipurpose Patient Care and CPR Infant Simulator, S117 Multipurpose Patient Care and CPR Pediatric Simulator, S157 Multipurpose Patient Care and CPR Pediatric Simulator, S575 Noelle®, S565 Noelle®, S560 Noelle®, S555 Noelle®, S550 Noelle®, S550. <NUM> Noelle®, S2200 Victoria®, S2220 Super Tory®, and/or other patient simulators.

In some embodiments, the virtual anatomic model <NUM> or <NUM> overlays features on the physical anatomic model <NUM> or <NUM> that are not ordinarily visible to the user (e.g., life-like animations of internal organs, internal lesions, etc.). In some embodiments, the virtual anatomic model <NUM> or <NUM> is synchronized with a physical response from the physical anatomic model <NUM> or <NUM> (e.g., changes to the fetal heart rate via the computing device <NUM> causing movement of the virtual fetal anatomy <NUM>, suprapubic pressure maneuvers applied to the physical anatomic model <NUM> or <NUM> will cause deflections to the virtual fetal anatomy <NUM>, etc.). In some embodiments, the AR system <NUM> further comprises a speaker (not shown) operable to add spatial sound to increase the realism of the simulated procedure (e.g., heart rate beep, speech from virtual anatomic model <NUM> or <NUM>, etc.) or to add audible cues to the ancillary virtual graphics <NUM>. In some embodiments, the virtual anatomic model <NUM> or <NUM> augments the external appearance of the physical anatomic model <NUM> or <NUM> using virtual elements (e.g., skin lesions, scars, wrinkles, etc.). In some embodiments, the virtual anatomic model <NUM> or <NUM> adds features that are absent from the physical anatomic model <NUM> or <NUM> (e.g., limbs, head, arms, torso). As a result, some or all of the physical anatomic model <NUM> or <NUM> is represented holographically by the virtual anatomic model <NUM> or <NUM>. In some embodiments, the ancillary virtual graphics <NUM> are added to the optic feedback <NUM> (i.e., via the optic feedback <NUM>-k) without occluding the user <NUM>'s field of view. In some embodiments, the AR headset device <NUM> includes one or more camera's to capture and record the user <NUM>'s gaze and position so that such information can be shared with the computing device <NUM> via the network <NUM> to debrief the user <NUM> after completion of a training exercise. In some embodiments, the AR system <NUM> is capable of displaying, via the virtual anatomic model <NUM> or <NUM> on the AR headset device <NUM>, an internal organ slice along <NUM> orthogonal planes.

Referring to <FIG>, with continuing reference to <FIG>, a method is generally referred to by the reference numeral <NUM>. The method <NUM> includes viewing a first optic feedback (e.g., the optic feedback 24aa or 24ca) that emanates from a physical anatomic model (e.g., the physical anatomic model <NUM> or <NUM>) and passes through the AR headset device <NUM> at a step <NUM>; tracking, using the tracking system <NUM>, a position and an orientation of the AR headset device <NUM> at a step <NUM>; receiving, using the computing device <NUM>, a first signal from the tracking system <NUM> relating to the position and the orientation of the AR headset device <NUM> at a step <NUM>; sending, using the computing device <NUM>, a second signal to the AR headset device <NUM> to cause the AR headset device <NUM> to overlay a second optic feedback (e.g., the optic feedback 24ab and 24b, or the optic feedback 24cb and 24d-f) on the first optic feedback at a step <NUM>, the second signal being based on the first signal; and viewing the second optic feedback at a step <NUM>, wherein the second optic feedback emanates from the AR headset device <NUM> and includes a virtual anatomic model (e.g., the virtual anatomic model <NUM> or <NUM>). In some embodiments, the second optic feedback further includes the ancillary virtual graphics <NUM> (e.g., the optic feedback <NUM>-k), the ancillary virtual graphics <NUM> including one or more of: the medical data <NUM>, the instructional steps <NUM>, the expert demonstrations <NUM>, the didactic content <NUM>, and the exigent circumstances <NUM>.

In some embodiments of the method <NUM>, the method <NUM> is accomplished using the physical anatomic model <NUM> and the virtual anatomic model <NUM>. In such embodiments, the method <NUM> further includes co-registering, using the computing device <NUM>, the virtual anatomy <NUM> with the manikin <NUM> to ensure an accurate spatial relationship therebetween, and the second signal is further based on the co-registering of the virtual anatomy <NUM> with the manikin <NUM>.

In other embodiments of the method <NUM>, the method <NUM> is accomplished using the physical anatomic model <NUM> and the virtual anatomic model <NUM>. In such embodiments, the method <NUM> further includes co-registering, using the computing device <NUM>, the virtual fetal anatomy <NUM> and/or the virtual maternal anatomy <NUM> with the maternal manikin <NUM> to ensure an accurate spatial relationship therebetween, and the second signal is further based on the co-registering of the virtual fetal anatomy <NUM> and/or the virtual maternal anatomy <NUM> with the maternal manikin <NUM>. In addition, or instead, the method <NUM> further includes tracking, using the tracking system <NUM>, a position and an orientation of the fetal manikin <NUM> relative to the maternal manikin <NUM>; co-registering, using the computing device <NUM>, the virtual fetal anatomy <NUM> with the fetal manikin <NUM> to ensure an accurate spatial relationship therebetween; and receiving, using the computing device <NUM>, a third signal from the tracking system <NUM> relating to the position and the orientation of the fetal manikin <NUM> relative to the maternal manikin <NUM>. In such embodiment(s), the second signal is further based on the third signal and the co-registering of the virtual fetal anatomy <NUM> with the fetal manikin <NUM>.

In an embodiment, as illustrated in <FIG>, an AR system is generally referred to by the reference numeral <NUM> and includes at least the components of the system <NUM>. More particularly, the AR system <NUM> includes the computing device <NUM>, the physical anatomic model <NUM> and/or <NUM>, and a plurality of AR headset devices <NUM><NUM>, <NUM><NUM>, <NUM><NUM>, and <NUM>i. The computing device <NUM> is connected to the network <NUM> and includes unified simulator control software <NUM> such as, for example Gaumard's UNI® software, to monitor and control the physiology of the physical anatomic model <NUM> and/or <NUM>, to monitor and control the AR headset devices <NUM><NUM>-i (and thus users' actions), and to support data-rich debriefing before, during, or after completion of a particular training scenario. The AR headset devices <NUM><NUM>-i are connected to the network <NUM> and include onboard tracking systems <NUM><NUM>, <NUM><NUM>, <NUM><NUM>, and <NUM>i, respectively, adapted to track the positions and orientations of the AR headset devices <NUM><NUM>-i in three-dimensional space and relative to the physical anatomic model <NUM> and/or <NUM>. Additionally, the AR headset devices <NUM><NUM>-i each include an onboard computing device <NUM>' separate from, but substantially similar to, the computing device <NUM>. The computing device <NUM>' of each AR headset device <NUM><NUM>-i runs an AR application <NUM> to facilitate a user's control of the various features and scenarios available to the AR headset devices <NUM><NUM>-i. In several embodiments, the AR system <NUM> also includes a computing device <NUM>" connected to the network <NUM> and adapted to run AR link software <NUM>, as will be described in further detail below.

In some embodiments, the unified simulator control software <NUM> may come preconfigured on a tablet PC and include a library of modifiable, prebuilt scenarios to save time and development resources. In some embodiments, the unified simulator control software <NUM> allows an instructor to control a variety of vital signs of the manikin to demonstrate a variety of patient presentations realistically. In some embodiments, the unified simulator control software <NUM> allows an instructor to create scenarios tailored to specific learning objectives and offer participants a wide range of standardized, repeatable learning events.

In some embodiments, the computing device <NUM> may be, include, or be part of a variety of computing devices; thus, as used herein the reference numeral <NUM> (without the suffixes (') or (")) may refer to one, or a combination, of the computing devices <NUM>' and <NUM>" described herein. In some embodiments, the AR headset device <NUM> may be, include, or be part of a variety of AR headset devices; thus, as used herein the reference numeral <NUM> (without the suffixes <NUM>, <NUM>, <NUM>, or i) may refer to one, or a combination, of the AR headset devices <NUM><NUM>-i described herein. In some embodiments, the tracking system <NUM> may be, include, or be part of a variety of tracking systems; thus, as used herein the reference numeral <NUM> (without the suffixes <NUM>, <NUM>, <NUM>, or i) may refer to one, or a combination, of the tracking systems <NUM><NUM>, <NUM><NUM>, <NUM><NUM>, and <NUM>i described herein.

In an embodiment, as illustrated in <FIG>, the AR application <NUM> on each AR headset device <NUM><NUM>-i includes a hub menu (not shown) presented to the user prior to the initiation of a particular scenario. The hub menu may be presented to the user on the screen of the AR headset device <NUM><NUM>, <NUM>, <NUM>, or i or on another display unit (e.g., a computer monitor, a television, etc.). The hub menu may include selectable icons for various AR simulations, depending on the particular type of training session desired, including, among others, an obstetric AR module <NUM> (shown in <FIG>), a trauma AR module (not shown), and a pediatric AR module (not shown). A graphical illustration of the obstetric AR module <NUM> is shown in <FIG> and includes selectable icons for a live mode <NUM>, a watch mode <NUM>, a tour mode <NUM>, and a setup mode <NUM>. The obstetric AR module <NUM>, for example, is designed to improve the student learning process by using overlays of obstetric anatomy (e.g., the virtual maternal anatomy <NUM> and/or the virtual fetal anatomy <NUM>), and animations to better visualize the mechanics of childbirth and various birthing complications, as will be described in further detail below. Such 3D animations have been proven to be better teaching aids than words and pictures in a book.

In an embodiment, as illustrated in <FIG>, the live mode <NUM> projects a holographic overlay of a 3D fetus and obstetric anatomy (e.g., the virtual maternal anatomy <NUM> and/or the virtual fetal anatomy <NUM>) on top of a physical simulator or manikin (e.g., the physical anatomic model <NUM>). The live mode <NUM> further includes a heads-up menu <NUM> having a variety of buttons selectable by the user such as, for example, a "hide menu" button <NUM>, a "return" button <NUM>, a "hide/show skin" button <NUM>, a "adjust view" button <NUM>, and a "code placement" button <NUM>. In the embodiment shown in <FIG>, the "hide/show skin" button <NUM> is toggled to the hide position such that the fetus and obstetric anatomy (e.g., the virtual fetal anatomy <NUM> and the internal virtual structures <NUM> if the virtual maternal anatomy <NUM>) are not visible. The "code placement" button <NUM> is operable to register the holographic animations to the physical manikin (e.g., the physical anatomic model <NUM>) in a manner that will be described in further detail below. The 3D animations may be updated periodically in accordance with the current state of the simulated labor scenario. Specifically, data may be communicated in real time between the AR headset devices <NUM><NUM>-i and the unified simulator control software <NUM> to update fetal rotations, fetal positions, and vital signs of both the mother and child to augment what is seen by the user in order to improve the educational experience, as will be described in further detail below. Because this transfer of data gives the users the current status of their simulated patient, warnings and directives may also be delivered to the users (and/or other participants) in the form of visual or audible cues. For example, as shown in <FIG>, the live mode <NUM> may include an emergency panel <NUM> capable of delivering a warning in the event that a simulated patient's heart rate should rise above a normal range (e.g., "heart rate is currently above normal parameters" "caution and assessment is advised").

In an embodiment, as illustrated in <FIG>, the watch mode <NUM> allows the user to play through animations of various childbirth scenarios and birthing complications along with teaching tips and maneuvers to facilitate childbirth in the event of a complication. An example selection menu <NUM> for the various birthing scenarios is illustrated in <FIG>. Examples of such childbirth scenarios include, but are not limited to, normal delivery <NUM>, shoulder dystocia <NUM>, breech delivery <NUM>, nuchal chord delivery <NUM>, and postpartum hemorrhage <NUM>. After a particular birthing scenario is selected from the selection menu <NUM> (e.g., when the user selects the type of birth he/she would like to view), the user is presented with a hologram of the manikin together with a birthing menu to control the scenario. An example birthing menu <NUM> for the normal delivery <NUM> is illustrated in <FIG>. In the embodiment shown in <FIG>, the birthing menu <NUM> includes a timeline <NUM> via which the user may select which stage of the normal delivery <NUM> to view, including a "begin descent" stage <NUM>, an "initial rotation" stage <NUM>, an "external rotation" stage <NUM>, an "entering birth canal" stage <NUM>, and a "delivery" stage <NUM>. The birthing menu <NUM> also includes buttons for controlling the simulation such as, for example a "hide menu" button <NUM>, a "return" button <NUM>, a "pause/play" button <NUM>, a "restart" button <NUM>, and an "adjust view" button <NUM>. The visualization of obstetric anatomy using life-size 3D holograms in the watch mode <NUM> helps to reinforce student learning.

In an embodiment, as illustrated in <FIG>, the tour mode <NUM> provides a tour of the various simulator features overlaid on a standing hologram <NUM> of the manikin (e.g., the maternal manikin <NUM>). Using an "adjust view" button <NUM>, the user can gaze at various parts of the hologram <NUM> to receive an overview of features pertaining to a particular part or parts of the hologram <NUM>. The tour mode <NUM> may also include various buttons and panels corresponding to the particular part of the hologram <NUM> at which the user is gazing. For example, <FIG> illustrates a scenario in which the user has selected the head of the standing hologram <NUM> of the manikin together with a "neurological" button <NUM>. In response to the user's selection of the "neurological" button <NUM>, the tour mode <NUM> displays an information panel <NUM> pertaining to neurological response features of the manikin (e.g., the maternal manikin <NUM>). The tour mode <NUM> may further include various other buttons and panels including, but not limited to, a skin tone panel <NUM> having buttons to select various skin tones to display on the standing hologram <NUM> of the manikin. The framework of the tour mode <NUM> is applicable to a wide variety of manikins (including, for example, the manikin <NUM>, the maternal manikin <NUM>, the fetal manikin <NUM>, and/or another manikin) and provides a teaching tool for students and observers to familiarize themselves with the various features of a particular manikin before, during, or after an interaction with the manikin.

In an embodiment, as illustrated in <FIG>, the setup mode <NUM> focuses on connectivity between the unified simulator control software <NUM> and the AR headset devices <NUM><NUM>-i. An example connectivity menu <NUM> including a "wireless setup" button <NUM> and a "Bluetooth setup" button <NUM> is illustrated in <FIG>. The user's selection of the "wireless setup" button <NUM> from the connectivity menu <NUM> presents the user with a scan box <NUM>, such as that illustrated in <FIG>, except that, rather than being blank, the scan box <NUM> is filled with the user's field of view (i.e., the field of view seen by the AR headset device <NUM><NUM>, <NUM>, <NUM>, or i). The scan box <NUM> can then be lined up by the user with a QR code <NUM> produced by the unified simulator control software <NUM> to establish a connection between the unified simulator control software <NUM> and the AR headset device <NUM><NUM>, <NUM>, <NUM>, or i, as shown in <FIG>. The QR code <NUM> generated by the unified simulator control software <NUM> contains all of the network settings required to establish such a connection (via, e.g., the network <NUM>).

In an embodiment, as illustrated in <FIG>, the obstetric AR module <NUM> utilizes object recognition software such as, for example, Vuforia™ object recognition software, to read and recognize a customized code <NUM> in order to place and orient holograms (e.g., the virtual anatomic model <NUM> and/or <NUM>) in the real world (e.g., relative to the physical anatomic model <NUM> and/or <NUM>). For example, this feature may be used in the live mode <NUM> to overlay a hologram over the physical manikin, and in the watch mode <NUM> to automatically place the hologram on a real-world surface (e.g., an examination surface). An example of the customized code <NUM> is illustrated in <FIG>. Moreover, an example of the placement of the customized code <NUM> on the manikin (e.g., the maternal manikin <NUM>) is illustrated in <FIG>. Specifically, the customized code <NUM> is placed in a particular location and orientation on the manikin so that, when the customized code <NUM> is scanned by the AR headset device <NUM><NUM>, <NUM>, <NUM>, or i, it will accurately register the hologram (e.g., the virtual anatomic model <NUM> and/or <NUM>) with the manikin (e.g., the physical anatomic model <NUM> and/or <NUM>), as shown in <FIG>. More particularly, <FIG> illustrates an embodiment in which the virtual anatomic model <NUM> (including the virtual maternal anatomy <NUM> and the virtual fetal anatomy <NUM>) is registered to the physical anatomic model <NUM> (including the maternal manikin <NUM> and the fetal manikin <NUM>) using the customized code <NUM>. In some embodiments, registration of the hologram with the manikin using the customized code <NUM> centers the hologram in relation to the manikin. The use of the customized code <NUM> to register the hologram with the manikin means that the user is not required to manually align the hologram with the manikin every time the obstetric AR module <NUM> is used (or the requirement for manual alignment is at least reduced).

In an embodiment, as illustrated in <FIG>, the unified simulator control software <NUM> has the ability to send questions and messages to the AR headset devices <NUM><NUM>-i as a group or to one specific AR headset device <NUM><NUM>, <NUM>, <NUM>, or i. Specifically, an instructor may send questions to a group of students, or may discretely send questions to an individual student to evaluate his/her understanding of a particular training scenario. For example, <FIG> shows an example question <NUM> being asked via the AR headset devices <NUM><NUM>, <NUM>, <NUM>, and/or i (i.e., "what is the patient's blood pressure") during the normal delivery <NUM> in watch mode <NUM>. For another example, <FIG> shows an example message <NUM> being delivered to the AR headset devices <NUM><NUM>, <NUM>, <NUM>, and/or i (i.e., "please double check the FHR, it might help you with your next diagnostic step") during the normal delivery <NUM> in watch mode <NUM>.

In an embodiment as illustrated in <FIG>, the unified simulator control software <NUM> includes an AR hub <NUM> configured to control at least some of the features in the AR application <NUM>, which application runs the AR headset devices <NUM><NUM>-i. As shown in <FIG>, the AR hub <NUM> provides feedback regarding the connectivity and current status of each AR headset device <NUM><NUM>-i connected to the unified simulator control software <NUM>. Moreover, the AR hub <NUM> allows an instructor to take full control of any connected AR headset device <NUM><NUM>, <NUM>, <NUM>, or i by selecting a "lock mode" button <NUM> (shown in the upper right corner of <FIG>). As a result, each AR headset device <NUM><NUM>-i is operable in two different modalities (i.e., instructor controlled and participant controlled). An example of feedback relating to the AR headset devices <NUM><NUM>, <NUM>, <NUM>, and/or i is shown in <FIG> and includes connectivity information 189a as well as icon feedback 189b of the current mode in which each of the AR headset devices <NUM><NUM>-i is operating (e.g., live mode <NUM>, watch mode <NUM>, tour mode <NUM>, or setup mode <NUM>). As shown in <FIG>, similarly to the obstetric AR module <NUM> accessible via the AR application <NUM>, the AR hub <NUM> includes a live mode <NUM>, a watch mode <NUM>, and a tour mode <NUM>. In the live mode <NUM> of the AR hub <NUM>, as shown in <FIG>, the instructor may select a particular scenario and run it live on both the manikin (e.g., the physical anatomic model <NUM> and/or <NUM>) and all of the connected AR headset devices <NUM><NUM>-i. In the watch mode <NUM> of the AR hub <NUM>, as shown in <FIG>, the instructor may select a video clip from the unified simulator control software <NUM> and have all connected AR headset devices <NUM><NUM>-i simultaneously watch an instructional video. Finally, in the tour mode <NUM> of the AR hub <NUM>, as shown in <FIG>, the instructor may provide a general overview of the manikin to all connected AR headset devices <NUM><NUM>-i by selecting different regions of the standing hologram <NUM> of the manikin (e.g., the maternal manikin <NUM>) and activating the selected regions on all connected AR headset devices <NUM><NUM>-i.

In an embodiment, as illustrated in <FIG> with continuing reference to <FIG>, the computing device <NUM>" may include the AR link software <NUM> to allow participants that do not have one of the AR headset devices <NUM><NUM>-i to still view the 3D animations of, for example, the live mode <NUM> during a particular scenario. An example of a graphical output <NUM> of the AR link software <NUM> is shown in <FIG>. In some embodiments, the AR link software <NUM> is a standalone desktop application including features similar to those of the AR application <NUM> (which runs the AR headset devices <NUM><NUM>-i). In some embodiments, the AR link software <NUM> establishes connectivity with the unified simulator control software <NUM> on the computing device <NUM> via Transmission Control Protocol/Internet Protocol ("TC/PIP") over the network <NUM>. In some embodiments, at least a portion of the AR link software <NUM> runs on the computing device <NUM>.

In an exemplary embodiment, as illustrated in <FIG>, the obstetric AR module <NUM> is configured to create real-time visualization of the fetus and inner obstetric anatomy (e.g., the virtual fetal anatomy <NUM> and the internal virtual structures <NUM> of the virtual maternal anatomy <NUM>) in lock step with the progress of a selected birthing scenario. For example, the fetus position and rotation may be animated together with the rest of the obstetric anatomy to complement a real-time birthing simulation using the manikin (e.g., the physical anatomic model <NUM>) and the unified simulator control software <NUM>. Specifically, obstetric animations are rendered in real-time on the AR headset devices <NUM><NUM>-i using a game engine <NUM> (e.g., Unity® 3D software) of the present disclosure. An example of the game engine <NUM> is shown diagrammatically in <FIG>, which illustrates the animation and rendering process utilized by the obstetric AR module <NUM>.

To ensure that animations are realistic, anatomically accurate, and renderable in real-time, the animations are first created offline using a professional animation software (e.g., Autodesk® Maya® 3D software, Blender™ 3D software, another 3D software, or the like), as indicated by the reference numeral <NUM>. Specifically, the offline animations are created using 3D animation software depicting the fetus in the correct pose (e.g., legs bent in fetal position, legs extended, etc.) corresponding to the position of the fetus during a particular stage of labor. Example orientations of the fetus during the birthing process are illustrated in <FIG>, together with the rest of the obstetric anatomy. In particular, as shown in <FIG>, the fetus undergoes both translation and rotation as the manikin (e.g., the maternal manikin <NUM>) progressed through labor. The animations created offline are then imported into the game engine <NUM>, as indicated by the arrow <NUM>.

The game engine <NUM> then modifies the imported animations to match the position and rotation of the fetus during the birthing scenario. Specifically, the modifications made to the imported animations by the game engine <NUM> are based on real-time feedback from the birthing simulator (e.g., the physical anatomic model <NUM> and/or the unified simulator control software <NUM>), as indicated by the arrow <NUM> in <FIG>, to impart the correct rotation to the pre-animated fetus pose. <FIG> depicts this two-step animation process. In particular, parts (A), (B), and (C) of <FIG> illustrate the first step of using the animation software to pre-animate the fetus with only translation and no rotation animation. Parts (D), (E), and (F) of <FIG> illustrate the second step of creating the desired rotation in real-time by rotating the pre-animated models using the game engine <NUM>. In some embodiments, this two-step animation process results in high visual fidelity, anatomical accuracy, and high frame rates (which are required for real-time rendering of AR images). The game engine <NUM> then outputs a real-time animation of the fetus and the obstetric anatomy based on the pre-animated fetus poses <NUM> and the real-time feedback <NUM> from the birthing simulator, as indicated diagrammatically in <FIG> by the reference numeral <NUM>. In some embodiments, the real-time animation of the fetus and the obstetric anatomy is then displayed on the AR headset device <NUM>.

Because the angular rotation of the fetus can vary from <NUM> to <NUM> degrees, to ensure high visual fidelity, in some embodiments, fetus poses are created for four different angular positions (in the offline 3D animation software <NUM>) as a function of labor progression. The various fetus poses for all other angular orientations are created on demand by the game engine <NUM> in real-time to match the feedback <NUM> from the birthing simulator. An example of this process is illustrated diagrammatically in <FIG>.

To ensure the placenta and umbilical cord remain attached to the fetus during rotation, the placenta is modeled with rotational and non-rotational parts. The rotational part allows the tip of the umbilical cord to stay attached to the fetus while the fetus is animated to rotate in the birth canal. An example of this process is illustrated diagrammatically in <FIG>. Specifically, part (A) of <FIG> shows an initial position of the fetus with the non-rotational part of the placenta being connected to the wall of the amniotic sac and the rotational part of the placenta (e.g., the umbilical cord) being connected to the fetus. In contrast, part (B) of <FIG> shows a rotated position of the fetus in which the tip of the umbilical cord remains attached to the fetus.

In an embodiment, as illustrated in <FIG>, a real-time video <NUM> of a simulation exercise may be overlaid with holographic AR animations <NUM> and/or other computer generated graphics. The AR animations <NUM> (shown in <FIG>) and graphics are rendered in real-time inside a real-time game engine <NUM> from a position and perspective that matches with the real-time video <NUM>, as shown in <FIG>. Specifically, the real-time video <NUM> of the simulation exercises is fed into the real-time game engine <NUM> together with real-time camera pose data (i.e., position and orientation), as indicated by the arrow <NUM> in <FIG>. Additionally, the AR animations and graphics generated by the game engine <NUM> (or a similar game engine) are fed into the real-time game engine <NUM>, as indicated by the arrow <NUM> in <FIG>. The real-time game engine <NUM> then creates a composite video of the real-time video <NUM> of the simulation exercise together with the AR animations and graphics, as indicated by the arrow <NUM> in <FIG>. Finally, the real-time game engine <NUM> sends the composite video to a video recording module <NUM>. Examples of the real-time video <NUM> of the simulation exercise, the AR animations <NUM> and graphics, and the composite video <NUM> created by the real-time game engine <NUM> are illustrated in <FIG>. Specifically, as shown in <FIG>, the real-time game engine <NUM> can be used to overlay the virtual anatomic model <NUM> on the physical anatomic model <NUM>. The real-time game engine <NUM> is equally suited to overlay the virtual anatomic model <NUM> on the physical anatomic model <NUM>.

In some embodiments, a plurality of instructions, or computer program(s), are stored on a non-transitory computer readable medium, the instructions or computer program(s) being accessible to, and executable by, one or more processors. In some embodiments, the one or more processors execute the plurality of instructions (or computer program(s)) to operate in whole or in part the above-described illustrative embodiments. In some embodiments, the one or more processors are part of the computing device <NUM> (or another computing device of the present disclosure), one or more other computing devices, or any combination thereof. In some embodiments, the non-transitory computer readable medium is part of the computing device <NUM>, one or more other computing devices, or any combination thereof.

Referring to <FIG> with continuing reference to <FIG>, an embodiment of the computing device <NUM>, <NUM>', and/or <NUM>" for implementing one or more embodiments of one or more of the above-described networks, elements, methods and/or steps, and/or any combination thereof, is depicted. The computing device <NUM>, <NUM>', and/or <NUM>" includes a microprocessor 14a, an input device 14b, a storage device 14c, a video controller 14d, a system memory 14e, a display 14f, and a communication device <NUM> all interconnected by one or more buses <NUM>. In some embodiments, the storage device 14c may include a floppy drive, hard drive, CD-ROM, optical drive, any other form of storage device and/or any combination thereof. In some embodiments, the storage device 14c may include, and/or be capable of receiving, a floppy disk, CD-ROM, DVD-ROM, or any other form of computer-readable medium that may contain executable instructions. In some embodiments, the communication device <NUM> may include a modem, network card, or any other device to enable the computing device to communicate with other computing devices. In some embodiments, any computing device represents a plurality of interconnected (whether by intranet or Internet) computer systems, including without limitation, personal computers, mainframes, PDAs, smartphones and cell phones.

The computing device can send a network message using proprietary protocol instructions to render 3D models and/or medical data. The link between the computing device and the display unit and the synchronization between the programmed state of physical manikin and the rendering data/3D model on the display unit of the present invention facilitate enhanced learning experiences for users. In this regard, multiple display units can be used simultaneously by multiple users to show the same 3D models/data from different points of view of the same manikin(s) to facilitate uniform teaching and learning, including team training aspects.

In some embodiments, one or more of the components of the above-described illustrative embodiments include at least the computing device <NUM>, <NUM>', and/or <NUM>" and/or components thereof, and/or one or more computing devices that are substantially similar to the computing device <NUM>, <NUM>', and/or <NUM>" and/or components thereof. In some embodiments, one or more of the above-described components of the computing device <NUM>, <NUM>', and/or <NUM>" include respective pluralities of same components.

In some embodiments, a computer system typically includes at least hardware capable of executing machine readable instructions, as well as the software for executing acts (typically machine-readable instructions) that produce a desired result. In some embodiments, a computer system may include hybrids of hardware and software, as well as computer sub-systems.

In some embodiments, hardware generally includes at least processor-capable platforms, such as client-machines (also known as personal computers or servers), and hand-held processing devices (such as smart phones, tablet computers, personal digital assistants (PDAs), or personal computing devices (PCDs), for example). In some embodiments, hardware may include any physical device that is capable of storing machine-readable instructions, such as memory or other data storage devices. In some embodiments, other forms of hardware include hardware sub-systems, including transfer devices such as modems, modem cards, ports, and port cards, for example.

In some embodiments, software includes any machine code stored in any memory medium, such as RAM or ROM, and machine code stored on other devices (such as floppy disks, flash memory, or a CD ROM, for example). In some embodiments, software may include source or object code. In some embodiments, software encompasses any set of instructions capable of being executed on a computing device such as, for example, on a client machine or server.

In some embodiments, combinations of software and hardware could also be used for providing enhanced functionality and performance for certain embodiments of the present disclosure. In an illustrative embodiment, software functions may be directly manufactured into a silicon chip. Accordingly, it should be understood that combinations of hardware and software are also included within the definition of a computer system and are thus envisioned by the present disclosure as possible equivalent structures and equivalent methods.

In some embodiments, computer readable mediums include, for example, passive data storage, such as a random access memory (RAM) as well as semi-permanent data storage such as a compact disk read only memory (CD-ROM). One or more illustrative embodiments of the present disclosure may be embodied in the RAM of a computer to transform a standard computer into a new specific computing machine. In some embodiments, data structures are defined organizations of data that may enable an embodiment of the present disclosure. In an illustrative embodiment, a data structure may provide an organization of data, or an organization of executable code.

In some embodiments, any networks and/or one or more portions thereof, may be designed to work on any specific architecture. In an illustrative embodiment, one or more portions of any networks may be executed on a single computer, local area networks, client-server networks, wide area networks, internets, hand-held and other portable and wireless devices and networks.

In some embodiments, a database may be any standard or proprietary database software. In some embodiments, the database may have fields, records, data, and other database elements that may be associated through database specific software. In some embodiments, data may be mapped. In some embodiments, mapping is the process of associating one data entry with another data entry. In an illustrative embodiment, the data contained in the location of a character file can be mapped to a field in a second table. In some embodiments, the physical location of the database is not limiting, and the database may be distributed. In an illustrative embodiment, the database may exist remotely from the server, and run on a separate platform. In an illustrative embodiment, the database may be accessible across the Internet. In some embodiments, more than one database may be implemented.

In some embodiments, a plurality of instructions stored on a non-transitory computer readable medium may be executed by one or more processors to cause the one or more processors to carry out or implement in whole or in part the above-described operation of each of the above-described illustrative embodiments of the AR system <NUM>, the method <NUM>, and/or any combination thereof. In some embodiments, such a processor may include the microprocessor 14a, and such a non-transitory computer readable medium may include the storage device 14c, the system memory 14e, or a combination thereof. Moreover, the computer readable medium may be distributed among one or more components of the AR system <NUM>, including, but not limited to, the physical anatomic model <NUM> or <NUM>, the AR headset device <NUM>, the tracking system <NUM>, the instrument <NUM>, or any combination thereof. In some embodiments, such a processor may execute the plurality of instructions in connection with a virtual computer system. In some embodiments, such a plurality of instructions may communicate directly with the one or more processors, and/or may interact with one or more operating systems, middleware, firmware, other applications, and/or any combination thereof, to cause the one or more processors to execute the instructions.

In a first aspect, the present disclosure introduces an augmented reality system, including a physical anatomic model; a display unit via which a user is adapted to receive a first optic feedback and a second optic feedback, the first optic feedback emanating from the physical anatomic model and passing through the display unit, and the second optic feedback emanating from the display unit and including a virtual anatomic model; a tracking system adapted to track a position and an orientation of the display unit; and a computing device adapted to: receive a first signal from the tracking system relating to the position and the orientation of the display unit, and send a second signal to the display unit to cause the display unit to overlay the second optic feedback on the first optic feedback, the second signal being based on the first signal. In some embodiments, the second optic feedback further includes ancillary virtual graphics, the ancillary virtual graphics including one or more of: medical data, instructional steps, expert demonstrations, didactic content, and exigent circumstances. In some embodiments, the physical anatomic model includes a manikin, the manikin including external physical features, and the external physical features including physical representations of one or more external characteristics associated with a natural human; and the virtual anatomic model includes virtual anatomy, the virtual anatomy including internal virtual structures, and the internal virtual structures including virtual representations of one or more internal characteristics associated with the natural human. In some embodiments, the computing device is further adapted to co-register the virtual anatomy with the manikin to ensure an accurate spatial relationship therebetween; and the second signal is further based on the co-registering of the virtual anatomy with the manikin. In some embodiments, the virtual anatomy further includes external virtual features, the external virtual features including virtual representations of one or more external characteristics associated with the natural human; and the external virtual features of the virtual anatomy simulate some external characteristics associated with the natural human, and the external physical features of the manikin simulate other external characteristics associated with the natural human. In some embodiments, the manikin further includes internal physical structures, the internal physical structures including physical representations of one or more internal characteristics associated with the natural human; and the internal physical structures of the manikin simulate some internal characteristics associated with the natural human, and the internal virtual structures of the virtual anatomy simulate other internal characteristics associated with the natural human. In some embodiments, the physical anatomic model includes a maternal manikin, the maternal manikin including first external physical features, and the first external physical features including physical representations of one or more external characteristics associated with a natural mother; and the virtual anatomic model includes virtual fetal anatomy, the virtual fetal anatomy including first external virtual features, and the first external virtual features including virtual representations of one or more external characteristics associated with a natural fetus. In some embodiments, the computing device is further adapted to co-register the virtual fetal anatomy with the maternal manikin to ensure an accurate spatial relationship therebetween; and the second signal is further based on the co-registering of the virtual fetal anatomy with the maternal manikin. In some embodiments, the physical anatomic model further includes a fetal manikin contained within the maternal manikin, the fetal manikin including second external physical features, and the second external physical features including physical representations of one or more external characteristics associated with the natural fetus; and the second external physical features of the fetal manikin simulate some external characteristics associated with the natural fetus, and the first external virtual features of the virtual fetal anatomy simulate other characteristics associated with the natural fetus. In some embodiments, the tracking system is further adapted to track a position and an orientation of the fetal manikin relative to the maternal manikin; the computing device is further adapted to: co-register the virtual fetal anatomy with the fetal manikin to ensure an accurate spatial relationship therebetween; and receive a third signal relating to the position and the orientation of the fetal manikin relative to the maternal manikin from the tracking system; and the second signal is further based on the third signal and the co-registering of the virtual fetal anatomy with the fetal manikin. In some embodiments, the virtual anatomic model further includes virtual maternal anatomy, the virtual maternal anatomy including internal virtual structures, and the internal virtual structures including virtual representations of one or more internal characteristics associated with the natural mother. In some embodiments, the computing device is further adapted to co-register the virtual fetal anatomy and the virtual maternal anatomy with the maternal manikin to ensure an accurate spatial relationship therebetween; and the second signal is further based on the co-registering of the virtual fetal anatomy and the virtual maternal anatomy with the maternal manikin. In some embodiments, the maternal manikin further includes internal physical structures, the internal physical structures including physical representations of one or more internal characteristics associated with the natural mother; and the internal physical structures of the maternal manikin simulate some internal characteristics associated with the natural mother, and the internal virtual structures of the virtual maternal anatomy simulate other internal characteristics associated with the natural mother. In some embodiments, the virtual anatomic model further includes virtual maternal anatomy, the virtual maternal anatomy including second external virtual features, and the second external virtual features including virtual representations of one or more external characteristics of the natural mother; and the second external virtual features of the virtual maternal anatomy simulate some external characteristics associated with the natural mother, and the first external physical features of the maternal manikin simulate other external characteristics associated with the natural mother.

In a second aspect, the present disclosure introduces a method, including viewing a first optic feedback that emanates from a physical anatomic model and passes through a display unit; tracking, using a tracking system, a position and an orientation of the display unit; receiving, using a computing device, a first signal from the tracking system relating to the position and the orientation of the display unit; sending, using the computing device, a second signal to the display unit to cause the display unit to overlay a second optic feedback on the first optic feedback, the second signal being based on the first signal; and viewing the second optic feedback, wherein the second optic feedback emanates from the display unit and includes a virtual anatomic model. In some embodiments, the second optic feedback further includes ancillary virtual graphics, the ancillary virtual graphics including one or more of: medical data, instructional steps, expert demonstrations, didactic content, and exigent circumstances. In some embodiments, the physical anatomic model includes a manikin, the manikin including external physical features, and the external physical features including physical representations of one or more external characteristics associated with a natural human; and the virtual anatomic model includes virtual anatomy, the virtual anatomy including internal virtual structures, and the internal virtual structures including virtual representations of one or more internal characteristics associated with the natural human. In some embodiments, the method further includes co-registering, using the computing device, the virtual anatomy with the manikin to ensure an accurate spatial relationship therebetween; wherein the second signal is further based on the co-registering of the virtual anatomy with the manikin. In some embodiments, the virtual anatomy further includes external virtual features, the external virtual features including virtual representations of one or more external characteristics associated with the natural human; and the external virtual features of the virtual anatomy simulate some external characteristics associated with the natural human, and the external physical features of the manikin simulate other external characteristics associated with the natural human. In some embodiments, the manikin further includes internal physical structures, the internal physical structures including physical representations of one or more internal characteristics associated with the natural human; and the internal physical structures of the manikin simulate some internal characteristics associated with the natural human, and the internal virtual structures of the virtual anatomy simulate other internal characteristics associated with the natural human. In some embodiments, the physical anatomic model includes a maternal manikin, the maternal manikin including first external physical features, and the first external physical features including physical representations of one or more external characteristics associated with a natural mother; and the virtual anatomic model includes virtual fetal anatomy, the virtual fetal anatomy including first external virtual features, and the first external virtual features including virtual representations of one or more external characteristics associated with a natural fetus. In some embodiments, the method further includes co-registering, using the computing device, the virtual fetal anatomy with the maternal manikin to ensure an accurate spatial relationship therebetween; wherein the second signal is further based on the co-registering of the virtual fetal anatomy with the maternal manikin. In some embodiments, the physical anatomic model further includes a fetal manikin contained within the maternal manikin, the fetal manikin including second external physical features, and the second external physical features including physical representations of one or more external characteristics associated with the natural fetus; and the second external physical features of the fetal manikin simulate some external characteristics associated with the natural fetus, and the first external virtual features of the virtual fetal anatomy simulate other characteristics associated with the natural fetus. In some embodiments, the method further includes tracking, using the tracking system, a position and an orientation of the fetal manikin relative to the maternal manikin; co-registering, using the computing device, the virtual fetal anatomy with the fetal manikin to ensure an accurate spatial relationship therebetween; and receiving, using the computing device, a third signal from the tracking system relating to the position and the orientation of the fetal manikin relative to the maternal manikin; wherein the second signal is further based on the third signal and the co-registering of the virtual fetal anatomy with the fetal manikin. In some embodiments, the virtual anatomic model further includes virtual maternal anatomy, the virtual maternal anatomy including internal virtual structures, and the internal virtual structures including virtual representations of one or more internal characteristics associated with the natural mother. In some embodiments, the method further includes co-registering, using the computing device, the virtual fetal anatomy and the virtual maternal anatomy with the maternal manikin to ensure an accurate spatial relationship therebetween; wherein the second signal is further based on the co-registering of the virtual fetal anatomy and the virtual maternal anatomy with the maternal manikin. In some embodiments, the maternal manikin further includes internal physical structures, the internal physical structures including physical representations of one or more internal characteristics associated with the natural mother; and the internal physical structures of the maternal manikin simulate some internal characteristics associated with the natural mother, and the internal virtual structures of the virtual maternal anatomy simulate other internal characteristics associated with the natural mother. In some embodiments, the virtual anatomic model further includes virtual maternal anatomy, the virtual maternal anatomy including second external virtual features, and the second external virtual features including virtual representations of one or more external characteristics of the natural mother; and the second external virtual features of the virtual maternal anatomy simulate some external characteristics associated with the natural mother, and the first external physical features of the maternal manikin simulate other external characteristics associated with the natural mother.

In a third aspect, the present disclosure introduces an apparatus, including a non-transitory computer readable medium; and a plurality of instructions stored on the non-transitory computer readable medium and executable by one or more processors, the plurality of instructions including: instructions that cause the one or more processors to track, using a tracking system, a position and an orientation of a display unit; instructions that cause the one or more processors to receive a first signal from the tracking system relating to the position and the orientation of the display unit; and instructions that cause the one or more processors to send a second signal to the display unit to cause the display unit to overlay a first optic feedback on a second optic feedback, the second signal being based on the first signal; wherein the second optic feedback emanates from a physical anatomic model and passes through a display unit, and the first optic feedback emanates from the display unit and includes a virtual anatomic model. In some embodiments, the first optic feedback further includes ancillary virtual graphics, the ancillary virtual graphics including one or more of: medical data, instructional steps, expert demonstrations, didactic content, and exigent circumstances. In some embodiments, the physical anatomic model includes a manikin, the manikin including external physical features, and the external physical features including physical representations of one or more external characteristics associated with a natural human; and the virtual anatomic model includes virtual anatomy, the virtual anatomy including internal virtual structures, and the internal virtual structures including virtual representations of one or more internal characteristics associated with the natural human. In some embodiments, the plurality of instructions further include instructions that cause the one or more processors to co-register the virtual anatomy with the manikin to ensure an accurate spatial relationship therebetween; and the second signal is further based on the co-registering of the virtual anatomy with the manikin. In some embodiments, the virtual anatomy further includes external virtual features, the external virtual features including virtual representations of one or more external characteristics associated with the natural human; and the external virtual features of the virtual anatomy simulate some external characteristics associated with the natural human, and the external physical features of the manikin simulate other external characteristics associated with the natural human. In some embodiments, the manikin further includes internal physical structures, the internal physical structures including physical representations of one or more internal characteristics associated with the natural human; and the internal physical structures of the manikin simulate some internal characteristics associated with the natural human, and the internal virtual structures of the virtual anatomy simulate other internal characteristics associated with the natural human. In some embodiments, the physical anatomic model includes a maternal manikin, the maternal manikin including first external physical features, and the first external physical features including physical representations of one or more external characteristics associated with a natural mother; and the virtual anatomic model includes virtual fetal anatomy, the virtual fetal anatomy including first external virtual features, and the first external virtual features including virtual representations of one or more external characteristics associated with a natural fetus. In some embodiments, the plurality of instructions further include instructions that cause the one or more processors to co-register the virtual fetal anatomy with the maternal manikin to ensure an accurate spatial relationship therebetween; and the second signal is further based on the co-registering of the virtual fetal anatomy with the maternal manikin. In some embodiments, the physical anatomic model further includes a fetal manikin contained within the maternal manikin, the fetal manikin including second external physical features, and the second external physical features including physical representations of one or more external characteristics associated with the natural fetus; and the second external physical features of the fetal manikin simulate some external characteristics associated with the natural fetus, and the first external virtual features of the virtual fetal anatomy simulate other characteristics associated with the natural fetus. In some embodiments, the plurality of instructions further include: instructions that cause the one or more processors to track, using the tracking system, a position and an orientation of the fetal manikin relative to the maternal manikin; instructions that cause the one or more processors to co-register the virtual fetal anatomy with the fetal manikin to ensure an accurate spatial relationship therebetween; and instructions that cause the one or more processors to receive a third signal from the tracking system relating to the position and the orientation of the fetal manikin relative to the maternal manikin; and the second signal is further based on the third signal and the co-registering of the virtual fetal anatomy with the fetal manikin. In some embodiments, the virtual anatomic model further includes virtual maternal anatomy, the virtual maternal anatomy including internal virtual structures, and the internal virtual structures including virtual representations of one or more internal characteristics associated with the natural mother. In some embodiments, the plurality of instructions further include instructions that cause the one or more processors to co-register the virtual fetal anatomy and the virtual maternal anatomy with the maternal manikin to ensure an accurate spatial relationship therebetween; and the second signal is further based on the co-registering of the virtual fetal anatomy and the virtual maternal anatomy with the maternal manikin. In some embodiments, the maternal manikin further includes internal physical structures, the internal physical structures including physical representations of one or more internal characteristics associated with the natural mother; and the internal physical structures of the maternal manikin simulate some internal characteristics associated with the natural mother, and the internal virtual structures of the virtual maternal anatomy simulate other internal characteristics associated with the natural mother. In some embodiments, the virtual anatomic model further includes virtual maternal anatomy, the virtual maternal anatomy including second external virtual features, and the second external virtual features including virtual representations of one or more external characteristics of the natural mother; and the second external virtual features of the virtual maternal anatomy simulate some external characteristics associated with the natural mother, and the first external physical features of the maternal manikin simulate other external characteristics associated with the natural mother.

It is understood that variations may be made in the foregoing without departing from the scope of the claims.

In several illustrative embodiments, the elements and teachings of the various illustrative embodiments may be combined in whole or in part in some or all of the illustrative embodiments. In addition, one or more of the elements and teachings of the various illustrative embodiments may be omitted, at least in part, and/or combined, at least in part, with one or more of the other elements and teachings of the various illustrative embodiments.

In several illustrative embodiments, while different steps, processes, and procedures are described as appearing as distinct acts, one or more of the steps, one or more of the processes, and/or one or more of the procedures may also be performed in different orders, simultaneously and/or sequentially. In several illustrative embodiments, the steps, processes and/or procedures may be merged into one or more steps, processes and/or procedures.

In several illustrative embodiments, one or more of the operational steps in each embodiment may be omitted. Moreover, in some instances, some features of the present disclosure may be employed without a corresponding use of the other features. Moreover, one or more of the above-described embodiments and/or variations may be combined in whole or in part with any one or more of the other above-described embodiments and/or variations.

In the foregoing description of certain embodiments, specific terminology has been resorted to for the sake of clarity.

Terms such as "left" and right", "front" and "rear", "above" and "below" and the like are used as words of convenience to provide reference points and are not to be construed as limiting terms.

Claim 1:
A method, comprising:
viewing a first optic feedback, which emanates from a physical anatomic model (<NUM>),
and passes through a display unit (<NUM>), wherein the physical anatomic model (<NUM>) comprises a maternal manikin (<NUM>) and a fetal manikin (<NUM>) contained within the maternal manikin (<NUM>);
tracking, using a tracking system (<NUM>), a position and an orientation of the display unit (<NUM>);
receiving, using a computing device (<NUM>), a first signal from the tracking system (<NUM>) relating to the position and the orientation of the display unit (<NUM>);
sending, using the computing device (<NUM>), a second signal to the display unit (<NUM>) to cause the display unit (<NUM>) to overlay a second optic feedback on the first optic feedback, the second signal being based on the first signal; and
viewing the second optic feedback,
wherein the second optic feedback emanates from the display unit (<NUM>) and comprises a virtual anatomic model (<NUM>) comprising a virtual fetal anatomy (<NUM>),
wherein the method further comprises:
tracking, using the tracking system (<NUM>), a position and an orientation of the fetal manikin (<NUM>) relative to the maternal manikin (<NUM>);
co-registering, using the computing device (<NUM>), the virtual fetal anatomy (<NUM>) with the fetal manikin (<NUM>) to ensure an accurate spatial relationship therebetween; and
receiving, using the computing device (<NUM>), a third signal from the tracking system (<NUM>) relating to the position and the orientation of the fetal manikin (<NUM>) relative to the maternal manikin (<NUM>);
wherein the second signal is further based on the third signal and the co-registering of the virtual fetal anatomy (<NUM>) with the fetal manikin (<NUM>).