Paper: MorphOcc: An Implicit Generative Model of Neuronal Morphologies (ICLR-2024)

Reviewer comment (Evaluation):
The method doesn't quite learn topology of neuronal dendrites. For example, in the latent traversals, the intermediate shapes are broken pieces and not valid dendritic trees (Fig 5). This is further exacerbated by post-processing.  
> To enhance the quality of our reconstructed meshes used for visualization, we remove small components using a greedy algorithm that progressively adds components until at least 75% of the vertices are included.

Author response:
Your points on the limited generalization are spot on, and we agree. However, this reflects the state of the field: There are currently no quantitative, unbiased methods to capture the 3d morphology of neurons. There are currently also no established implicit methods that can handle generation of entire distributions of complex 3d shapes. Established methods from 3d computer vision either perform much worse than ours at generating fine-grained structures (Fig. 3) or do not address generalization at all, but overfit to individual scenes/objects. While there is obviously room for improvement, our approach clearly improves over the existing state of the art both qualitatively (Fig. 3) and quantitatively (Table 2). Hence, we consider it a valuable contribution.

Reviewer comment (Theory):
The volume bound has to be selected a-priori for the dataset. This approach seems to not be extensible for non-local morphologies (e.g. considering long range axons would require looking at the entire brain volume).

Author response:
That’s correct, but it seems to be necessary for any method. To model a highly non-local object, we need to consider a large volume which is going to be very sparse. Are we missing something? Can you give an example for a method where this wouldn’t hold?

Reviewer comment (Experiments):
It seems that the network has to be evaluated on points spread throughout the entire volume to create the iso-surface. This seems quite expensive to generate a single neuron's dendritic morphology.

Author response:
That’s also correct, but there are standard and efficient ways of doing so (e.g. marching cubes). The big advantage is that we don’t have to store and evaluate this entire volume during training, enabling us to work with such a large volume in the first place.

Reviewer comment (Experiments):
The following statement seems to be an over-reach, since no tests are performed with out-of-distribution samples.  
> Moreover, this process serves as a testament to the model’s generalization capabilities, as it effectively handles out-of-distribution samples.

Author response:
That’s a wording error on our side. Thanks for catching it! They are not out-of-distribution samples, but held-out samples not seen during training. We will fix it.

Reviewer comment (Novelty):
Capturing connectivity (e.g. tree structure) should be a crucial ingredient in generative models for morphology. This is not captured in the current model. There isn't methodological novelty (Pointnets and SIREN are off-the-shelf components).

Author response:
You are right that PointNet encoder and SIREN are not novel. The novelty lies in the realization that the problem is not the architecture but the structure of the training data: For such fine-grained structures as neurons are, the established strategies for sampling off-surface points are insufficient, which is why we propose the hard negative sampling along with a learning curriculum, which we show to improve performance substantially.

Reviewer comment (Experiments):
If one had 10x or even 100x as many morphologies, how would this method accommodate this? Is it mainly through increasing complexity of the decoder?

Author response:
Yes, with more data one could increase the capacity in the encoder and decoder as well as increase the latent dimension to capture more details.

Reviewer comment (Evaluation):
The soma position and coarse dendritic density are very suggestive of the cell types. In Fig 2, the representations seem organized by that. Is this a fair assessment? If so why is classification a good test of representations?

Author response:
Please note that for all neurons the soma is put at the origin (0,0,0) of the coordinate system. Hence, the absolute soma position is not available to the model. While for some neuron types the rough dendritic morphology is obviously helpful, that’s not the case for all of them (e.g. layer 6 neurons which are often short and similar to layer 2/3 or 4 neurons).

Reviewer comment (Experiments):
While not explicitly so, the volume is the same for the entire dataset, and the soma / origin of the dendrites contains layer information. Do the authors agree?  
> Some of the layer 6 cells are more dispersed as they morphologically resemble inhibitory and more superficial cells, and the model is not provided with the laminar location.

Author response:
Please note that for all neurons the soma is put at the origin (0,0,0) of the coordinate system. Hence, the absolute soma position is not available to the model. While for some neuron types the rough dendritic morphology is obviously helpful, that’s not the case for all of them (e.g. layer 6 neurons which are often short and similar to layer 2/3 or 4 neurons).

Reviewer comment (Experiments):
A comparison of classification results with something simple e.g. density representation / PCA of the neuron point cloud (with similar bounding boxes and normalization as chosen for the model here) would help assess the improvements better.

Author response:
Regarding Q5: As you suggested, we compared our classification results with a density representation of the neuron point cloud. We devised vertical histograms of the normalized and centered point clouds. By segmenting the y-axis into 64 bins, we generated an embedding of equivalent dimensionality as generated by our model. The results of density-based classification (99% I/E, 75% cell type, 88% layer) are comparable to those achieved by our model (99% I/E, 74% cell type, 84% layer). 

Note that this baseline is anything but trivial, as it uses the domain knowledge that the vertical density distribution contains a lot of information about the cell type. Our method has learned to extract this information without us explicitly providing it. Nevertheless, it suggests that our current (Pointnet-based) encoder might be too strong a bottleneck not capturing much of fine structure, likely leading to memorization in the decoder and impede generalization.
 
We conclude that the contribution of our work is the strategy for achieving fine-grained reconstruction, but there remains room for improvement in the development of a more powerful encoder that captures more of the fine structure in the embeddings and improves generalization.

Reviewer comment (Experiments):
Only a single EM dataset is used in the paper. Have you considered other proofread volume EM datasets, particularly ones that are from different species, e.g. fruit fly?

Author response:
Dataset: These datasets are large, complex and so far not standardized, making it a substantial engineering effort to work with additional datasets from different labs/consortia. What added benefit would you expect from a second dataset? We believe the current dataset shows both the challenges of the type of data and the progress we have made.

Reviewer comment (Evaluation):
The interpolation results are not convincing in that the generated meshes seem to have disconnected components and thus not represent valid neurons.

Author response:
Interpolation: We agree that the interpolation results are not yet satisfying in absolute terms. However, this reflects the state of the field: There are currently no quantitative, unbiased methods to capture the 3d morphology of neurons. There are currently also no established implicit methods that can handle generation of entire distributions of complex 3d shapes. Established methods from 3d computer vision either perform much worse than ours at generating fine-grained structures (Fig. 3) or do not address generalization at all, but overfit to individual scenes/objects. While there is room for improvement, our approach clearly improves over the existing state of the art both qualitatively (Fig. 3) and quantitatively (Table 2). We are happy to state this limitation more clearly in the revision. Would you agree with this assessment? If not, where do you disagree or what would you suggest we do?

Reviewer comment (Experiments):
Have you evaluated the impact of a larger latent dimension size?

Author response:
Regarding your second and third question on the size of the latents and the ratio of sample points, we have started to run additional experiments and will update here as soon as we have the results.

Reviewer comment (Novelty):
The reliance on existing deep learning architectures like the PointNet encoder and SIREN decoder, without significant modifications or enhancements, raises concerns about technical novelty, especially considering the high standards expected for technical novelty in ICLR.

Author response:
Technical novelty: You are right that PointNet encoder and SIREN are not novel. The novelty lies in the realization that the problem is not the architecture but the structure of the training data: For such fine-grained structures as neurons are, the established strategies for sampling off-surface points are insufficient, which is why we propose the hard negative sampling along with a learning curriculum, which we show to improve performance substantially.