# Development of an Augmented Reality Surgical Trainer for Minimally Invasive Pancreatic Surgery
## 2 Materials and Methods
### Pancreas Reconstruction and Setup  
For a detailed representation of the soft tissue, the organ was reconstructed from CT scans. The process was carried out in Slicer 3D [21], where the targeted organ was segmented using the Total Segmentator module [22]. This method separates the targeted organ by analyzing color differences between pixels in the CT slices. After segmentation, the organ was reconstructed, and the model was exported as a 3D object. The final object was then imported into Unity, where custom textures were applied to give it a more accurate appearance of porcine tissues. The workflow for patient-specific pancreas modeling consists of CT segmentation of the organ, followed by 3D reconstruction, and mesh representation for simulation purposes, as illustrated in Figure 6.  
Figure 6: Workflow for patient-specific pancreas modeling. To achieve an approximation of the biomechanical properties of the porcine pancreas in Unity, several techniques were implemented to simulate its key characteristics, such as its Young's modulus, Poisson's ratio, and viscoelastic behavior. A spring-based deformation model was developed to mimic the pancreas' elastic modulus, which varies in the range of 1.40 \(\pm\) 0.47 kPa [23]. The spring constants were set to represent a similar stiffness range, allowing the organ to deform once forces were applied but return to its original shape when the force was removed. Poisson's ratio of the pancreas, typically between 0.45 and 0.5 [24], was incorporated into the deformation model by simulating volume conservation during compression. In Unity, the Rigidbody component was utilized to apply forces along the \(z\)-axis, mimicking compression. To prevent lateral movement, the rigid body's constraints, FreezePositionX and FreezePositionY, were selected. To replicate the tissue's resistance to lateral expansion under compression, this restricted expansion in the x and y directions maintained the volume constant throughout deformation. To detect collisions, a mesh collider was added, directing the deformation-based mesh vertices. When forces are applied, the positions of the vertices are adjusted to reflect interactions with the surgical instruments, allowing for realistic, local deformations based on the collisions. A maximum deformation failure threshold of 1 N was established, beyond which the mesh would undergo permanent modifications. If the organ was subjected to forces exceeding this threshold, the mesh would either rupture or deform permanently. The deformation limitations were maintained by simulating tissue failure when the 1 N force threshold was reached and using custom scripts to measure strain over time. Applying increasing damp pressures in parallel with the deformation rate was necessary to simulate the pancreas' viscoelastic characteristics (relaxation duration between 0.1 and 2 s). These forces imitated the relaxing behavior of soft tissues by simulating their time-dependent resistance to deformation, where the organ resists rapid motions more than slow ones. Finally, time-dependent pressures were applied to replicate stress relaxation behavior [24]. To replicate the stress relaxation observed in the pancreas during compression testing, the stress in the organ was set to decrease over time. The parallel robot described in Section 2.2 was also imported into Unity. The robot was initially designed in NX Siemens, exported as STEP files, and converted to FBX files (the format supported by Unity). By utilizing the Configurable Joint in Unity and considering both its active and passive joints, the robot was designed to function with both the haptic device and in a collaborative mode, allowing manual control. The latter feature was effectively implemented using the HoloLens 2 device. The integration of the robot as a hologram into the operating room (Figure 7) provides a powerful tool for both the developers and the medical professionals. By using HoloLens 2, the robot is projected as a 3D hologram, enabling real-time visualization and interaction during surgical procedures.