# ConceptVAE: Self-Supervised Fine-Grained Concept Disentanglement from 2D Echocardiographies
## 4 Latent Space and Qualitative Analysis  
Upon convergence, the pre-trained model can be qualitatively analysed by examining the inferred concept probability maps for test images. A straightforward method to implement this involves selecting the most likely concept at each grid location (\(c_{ij}=\arg\max p_{ij}(c)\)) and overlaying the up-sampled concept indices grid onto the initial input images, as in Figure 3. The probability of the most likely concept \(p(c_{ij})=\max p(c)\) at each location \(i,j\) can be incorporated in the visualisations. By examining a random selection of samples illustrated in Figure 3, we can make the following initial observations:  
* The prior constraint, which requires regions outside the cone to be modeled solely by the first concept (i.e., the background concept at index 0) is generally respected. Exceptions occur at grid locations in the cone's proximity, particularly at the boundaries between the cone and the background. As these are transition regions, they are not particularly concerning, since the model's confidence is expected to be low for such regions. * Certain concepts are specialized for specific anatomical structures. For example, concept \(c_{11}\) models blood pools within the cone, concept \(c_{1}\) represents the Left Ventricle (LV) free wall on the right hand size of the cone, concepts \(c_{5}\) and \(c_{7}\) correspond to septum walls, and concept \(c_{6}\) covers the right-heart side of the cone, among others. * Certain concepts, such as e.g., \(c_{13}\) and \(c_{14}\) appear more isolated and spanning a single grid location. By qualitatively assessing multiple input samples, we hypothesise these concepts encode information about the local anatomical shapes of nearby larger concept islands. It appears these concepts have larger confidence assigned to them than the average confidence inside larger concept islands. We term them _modifier_ concepts. To qualitatively evaluate the impact of modifier concepts, the greedy concept map of the middle image of Figure 3 is modified in two ways, by swapping 2 modifier and 2 normal concepts: first, (i) the modifier concepts \(c_{13}\) and \(c_{14}\) are swapped and the image is reconstructed without any style component (\(x_{style}:=0\)); and (ii) starting from the greedy map, concepts \(c_{5}\) and \(c_{1}\) are now swapped and the image is reconstructed in the same manner (with \(x_{style}:=0\)). The effects are illustrated in Figure 4: in the former case only minor shape modifications are observed around the grid locations where concept swaps were done. In the latter case, the effect is more significant, as it appears that the LV free wall changed place with the septum. While modifier concepts seem to function primarily in a styling role, it is important to note that the _Feature Decoder_ block processes \(k\times k\) regions of adjacent concept locations  
Figure 4: Effect of concept swapping. The left image is the reconstruction based only on the greedy concept map (with \(x_{style}:=0\)). The middle reconstruction illustrates the effect of swapping 2 modifier concepts, while the right reconstruction illustrates big changes induced by swapping two anatomy-specific concepts. Figure 3: Concept maps for three randomly sampled inputs. The 16\(\times\)-stride concept grid is upsampled to the original image size. The indices of the most likely concept for each grid location are displayed in red at the bottom-left of each location. The grid is color-coded according to concept indices for better visualisation. to reconstruct the low-level image features \(x_{stem}\). This means that neighboring concepts cooperate to form larger and more complex anatomical structures. Modifier concepts are not devoid of semantic meaning, as our experiments showed that replacing a specialized anatomical concept like \(c_{1}\) with a modifier concept still yields similar reconstructions, albeit with slight alterations in shape and/or region brightness patterns.