# MUSIC: Cardiac Imaging, Modelling and Visualisation Software for Diagnosis and Therapy
## 4 Functionalities
### Artificial Intelligence  
Deep Learning-based algorithms developed by our team have been recently included in Python script pipelines and further used in the MUSICardio application. These AI algorithms included in the scripts can improve the accuracy of cardiac segmentation and speed up the analysis and computations, resulting in a less invasive patient diagnostic, a more precise treatment, as well as a better therapy outcome. The Python support in MUSICardio pipeline allows us to access an unlimited number of calculation libraries, including the AI libraries such as: TensorFlow, PyTorch, and scikit-learn. The pipeline script also has several extended front-end controls of the MUSICardio interface, which are able to facilitate the visualisation of processing steps undertaken during the AI project. The ability to display different type of images and resulted computations is crucial in building a coherent and easy-to-understand pipeline starting from the input  
Figure 10: Distribution of the electrical potential on the heart surface. Simulated ground truth solution (**a**), MFS solution (**b**) and FEM solution (**c**). Figure 9: Examples of ECGI-oriented segmentations and mesh generation. (**a**): A 2D cross-section view of a segmented geometry showing the heart (dark red), lungs (green), liver (yellow) and the remaining tissue (red). (**b**): A 3D mesh of a torso volume. image to the final Deep Learning (DL) model prediction. However, the majority of the AI project steps are already built in Python, and these can be easily integrated into the main pipeline. One of the AI pipelines developed by our team automatically predicts the risk of ventricular arrhythmia (VA) from 3D CT image-based models, as proposed in [14]. This particular pipeline consists of several image processing steps including segmentation of CT images, visualisation of short-axis view orientation, wall thickness calculation, and 2D bullseye map flattening. The bullseye thickness map is then used as input to a VA classification network, after which the classification's attention map is calculated from the prediction score using a gradient back propagation algorithm. The automatic image segmentation is performed using a pre-trained Dual-UNet segmentation network, while the VA classifier network consists of a conditional variational autoencoder and a fully connected classifier model. The two DL networks and all matrix calculations can be integrated directly in the pipeline Python script. The complete image processing pipeline is presented in Figure 11.  
The pipeline ends with the VA classification from the 2D wall thickness map using a conditional variational autoencoder and a fully connected classifier model. A so-called \"attention map\" is also generated from the gradient back propagation of the classification score. The thickness map, attention map and classification score can be displayed at the end of the pipeline, as shown in Figure 12.  
The pipeline that was implemented and designed to be used case by case was not adapted for the processing of multiple inputs/outputs required for the development stage of the AI models. Therefore, both AI models were built, trained and validated outside of MUSICradio using the Python environment and packages, including TensorFlow (Deep Learning algorithms) and SimpleITK, Numpy and VTK (image processing). The segmentation network was built using two consecutive 3D UNet models. The first model was used to coarsely segment the heart regions, which was used to crop the region of interest to be used as input in the second segmentation model for refine segmentation. The model achieved a Dice score of 0.90 for left ventricle wall segmentation on the validation dataset. Similarly, the post-infarct VA prediction was trained using a retrospective dataset of 600 patients (27.5% VA). The validation on the test dataset showed the prediction accuracy  
Figure 11: Automatic pipeline for LV (left ventricular) wall thickness map extraction and VA prediction. The arrows illustrate the background calculations and AI predictions.