Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The manuscript proposed an unsupervised image registration algorithm based on the optimal transfer scheme, solving an unbalanced fused Gromov-Wasserstein optimization problem. The algorithm was evaluated on an fMRI data registration task to test whether the correspondence between fMRI signals can be improved after the registration. 
strengths_and_weaknesses: Effective registration from one individual image to a common space, as well as from one individual to another individual, is highly needed in medical image analysis. Although various registration algorithms have been developed, the proposed fused Gromov-Wasserstein optimal transfer algorithm with unbalanced matching is a novel and plausible way to solve this task in an unsupervised approach. The capability of considering both the image itself (i.e., the mesh) and features defined on the image (e.g., the activation map used in this work) is much needed especially for the multi-modal image fusion. The methodology of this work is clearly written, and the overall quality of this manuscript is good. 

The major issue of this work is that its evaluation scheme needs to be improved, see questions below.

questions: The proposed FUGW framework was tested on a cortical surface registration task on the activation maps and evaluated by the correlation of fMRI signals before/after the alignment. It will be great if it can be evaluated by anatomical features of the cortical surface (e.g., correspondence between the major gyri/sulci locations), which can be done by a small amount of human annotation. This is specifically important as in the paper, the fMRI-derived activation value is used as the feature to perform the registration, then the registration was evaluated by the same fMRI signal’s correlation. Thus, the improved correlation after the registration is somehow expected, as the vertex with a similar response to stimuli (reflected in the activation value) would usually have a similar fMRI signal. 

As the author mentioned in line 220, the selection of α is important for balancing anatomical validity and registration effectiveness. Is there a way to estimate this hyperparameter in a more intuitive way (e.g., by enforcing voxels in the same gyri/sulci would not move to another gyri/sulci)?

More details of the FUGW framework need to be clarified to understand its mechanism and apply it for other tasks:
1)	 How w^s and w^t in Eq. 1 are defined? The author mentioned that “We also assign the distribution w^s∈R^n_+ on the source vertices, which we interpret as their relative importance in the mesh” but more details are needed.  
2)	For the Barycenter estimation, how the mesh of the Barycenter is defined/updated (e.g., number of vertices) in each iteration? As each subject has a different number of vertices, it is curious that whether the Barycenter has a fixed number of vertices or varying vertices at each iteration. 

In section 4.1, the first experiment, the author mentioned “Aligning subjects on a fixed mesh”, does the “fixed mesh” here indicate the target of registration which is the template? 
limitations: N/A, this works only developed an image registration algorithm.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors propose an optimal transport based registration method for meshes with multi-variate features, a principle use-case of which is neuro-imaging (cortical mesh alignment). They construct a "Gromov-Wasserstein" loss which is the usual transport distance term to penalize large deformations w.r.t. the mesh geodesic, which they use alongside a "Wasserstein" loss, which is the transport matching term. Overall the transport method is defined for an "unbalanced" transport problem, where the domains need not match and where restrictions may be placed on transport to/from specific regions.

Experiments are presented with both synthetic and real data, using subsets of the Individual Brain Charting dataset (composed of a variety of task fMRI and the standard anatomical scans). Comparisons are made with Multisurface Matching (MSM) (Robinson 2014), as well as a template analysis (barycenters).
strengths_and_weaknesses: Strengths:  
* This paper is well written, and enjoyable to read.  
* The treatment of the transport problem is well done, and the implementation of a standard solution method (scaling) to this particular problem seems sound.  

Weaknesses:
* The method requires a dense interaction matrix of $V^2$ size, where $V$ is the number of vertices on the mesh, limiting this method to something on the order of 10k vertices. (Though, to the authors credit, considerations were made here.)   
* The baseline method is untuned.  

Comment:
* The technical innovation is limited to a specific application; this is not disqualifying, but it is limiting in "significance" to the NeurIPS community relative to, say, a conference or journal from the application domain (e.g., IEEE TMI).  
questions: My primary concern is with evaluation. While evaluating non-linear inter-subject registrations is always difficult, here we have chosen to ignore landmarks and focus on functional correlation of a hold-out set. While this is not unreasonable on its own, it is in the optimization desiderata themselves, and also in the hyper-parameter tuning objective. How can we know that these registrations are reasonable? Hold out correlation is one solution, but surely these trials are not fully independent (they do come from the same heads and, importantly, the same arterial structures). Could these comparisons be made using, e.g., myelin patterning, or its proxy by T1/T2 ratio?  

Another issue is that the comparison is made against MSM, which has default parameters set for an HCP-like dataset. Were the same search-grid to be made for MSM, would we see a similar increase? It appears to do nothing to the correlations; perhaps this is actually the case, but it seems like a very weak baseline.  

The authors specifically avoid diffeomorphic constraints. Given this, and average displacement on the order of 1-2cm (Fig 4), do we find extreme topological changes? Extreme geometric changes? Sulci to gyri?

Other comments:
* Robinson 2014 is not cited in the actual text of the paper; it may be helpful to readers during the introduction of MSM to do so.  
* Very little time is spent on the solution method, or the mesh-reduction method. While understandably space is constrained, I feel it would be helpful to have more explanation in the main text.  
limitations: None.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: A common problem in the use of brain imaging data is that individual brains differ in both geometry and function, requiring some form of alignment before comparison. The authors propose a novel method for aligning different brains, FUGW, which uses a combination of distance losses to respect functional, geometric, and even fundamental region differences between different brain images
strengths_and_weaknesses: Strengths: 

The problem statement and motivations are clear. The problem of accurate brain alignment is interesting both for current human neuroscience research and potential future cross-species comparisons.
The paper is well organized and clearly written. Toy examples are given to demonstrate the differences of their method
The method itself seems technically sound, with appropriate references to past work and a clear intuition of what each term does. 
The numerical experiments are well-designed and explained, and they demonstrate a marked advantage of the proposed method in both quantitative and qualitative terms.

Weaknesses:
I would have liked to see the impact of these brain alignments on downstream experiments/tasks. 
I would also be interested in computational comparisons; how much more difficult is it to do this vs MSM? In experiment 2, it was necessary to reduce the scale of the individual brain meshes to run at all (though admittedly, the other methods couldn’t even be applied)

questions: What would the impacts of these improvements be on downstream experiments/comparisons? There’s clearly a numerical advantage, but does that translate to application or understanding?

Do you have any examples of actually aligning brain images where one lacks certain region(s)? The toy sphere example sort of shows this

limitations: The technique may not scale well computationally, though that could more be an issue of the size of the images themselves.
review_title: null

Meta-review: metareview: This paper uses optimal transport for aligning cortical surfaces based on the similarity of their functional signatures under different stimulations. The paper is well written, and the experimental setup is sound. The authors added experiments and clarifications to address reviewers' comments and concerns. The reviewers provided a consensus accept rating for this paper. 

confidence: Certain
Decision: Accept