Title: Coupled Reconstruction of Cortical Surfaces by Diffeomorphic Mesh Deformation
Venue: NeurIPS 2023
Accepted: yes
Keywords: Brain MRIs

cortical surface reconstruction

deep learning
Abstract: Accurate reconstruction of cortical surfaces from brain magnetic resonance images (MRIs) remains a challenging task due to the notorious partial volume effect in brain MRIs and the cerebral cortex's thin and highly folded patterns. Although many promising deep learning-based cortical surface reconstruction methods have been developed, they typically fail to model the interdependence between inner (white matter) and outer (pial) cortical surfaces, which can help generate cortical surfaces with spherical topology. To robustly reconstruct the cortical surfaces with topological correctness, we develop a new deep learning framework to jointly reconstruct the inner, outer, and their in-between (midthickness) surfaces and estimate cortical thickness directly from 3D MRIs. Our method first estimates the midthickness surface and then learns three diffeomorphic flows jointly to optimize the midthickness surface and deform it inward and outward to the inner and outer cortical surfaces respectively, regularized by topological correctness. Our method also outputs a cortex thickness value for each surface vertex, estimated from its diffeomorphic deformation trajectory. Our method has been evaluated on two large-scale neuroimaging datasets, including ADNI and OASIS, achieving state-of-the-art cortical surface reconstruction performance in terms of accuracy, surface regularity, and computation efficiency.

Coupled Reconstruction of Cortical Surfaces by Diffeomorphic Mesh Deformation

Abstract

Accurate reconstruction of cortical surfaces from brain magnetic resonance images (MRIs) remains a challenging task due to the notorious partial volume effect in brain MRIs and the cerebral cortex's thin and highly folded patterns. Although many promising deep learning-based cortical surface reconstruction methods have been developed, they typically fail to model the interdependence between inner (white matter) and outer (pial) cortical surfaces, which can help generate cortical surfaces with spherical topology. To robustly reconstruct the cortical surfaces with topological correctness, we develop a new deep learning framework to jointly reconstruct the inner, outer, and their in-between (midthickness) surfaces and estimate cortical thickness directly from 3D MRIs. Our method first estimates the midthickness surface and then learns three diffeomorphic flows jointly to optimize the midthickness surface and deform it inward and outward to the inner and outer cortical surfaces respectively, regularized by topological correctness. Our method also outputs a cortex thickness value for each surface vertex, estimated from its diffeomorphic deformation trajectory. Our method has been evaluated on two large-scale neuroimaging datasets, including ADNI and OASIS, achieving state-ofthe-art cortical surface reconstruction performance in terms of accuracy, surface regularity, and computation efficiency.

1 Introduction

The analysis of the cerebral cortex using magnetic resonance imaging (MRI) is crucial in understanding neurodegenerative diseases [9, 43] and psychological disorders [42]. Since the cerebral cortex is a thin (a few millimeters thick) and highly folded sheet between the inner (white matter: WM) and outer (pial) surfaces, voxel-based segmentation methods cannot accurately capture its complicated morphology [17]. Instead, triangular meshes have been widely used for cortical surface reconstruction (CSR) [14] in order to accurately measure cortical thickness, volume, and gyrification. Although well-established methods for CSR (e.g., FreeSurfer [17], BrainSuite [48]) can produce promising results, they often require significant computational resources (e.g., 6h/subject [17]) and may necessitate manual editing in order to attain sub-voxel precision.

Recently, deep learning (DL) methods have achieved significant improvement in CSR in terms of both accuracy (sub-voxel error) and efficiency (orders of magnitude faster) [10, 13, 19, 20, 26, 30, 31, 41, 47, 59]. These DL methods can be broadly classified into two categories according to the representation of the output surfaces. (I) Implicit surface representation, such as signed distance function [13, 19], occupancy filed [13], and level set [41], can be predicted by neural networks, and cortical surface is then generated by a marching cubes algorithm [27]. (II) Explicit surface reconstruction methods take a coarse or fine initial mesh as input and directly predict a target mesh [10, 20, 26, 30, 31, 59]. A detailed comparison of the existing DL-based CSR methods is summarized in Table 1.

However, the existing DL-based CSR methods are subject to limitations. First , the interdependence between the inner and outer cortical surfaces is generally ignored, and therefore separate or multistage DL models are typically trained to reconstruct both the inner and outer cortical surfaces and intersections between them may occur. Even if both the inner and outer surfaces can be reconstructed simultaneously [10], they are loosely combined with no topological constraints. Second , complex DL architectures are commonly used, with separate learning of image and surface/vertex features using both convolutional neural networks (CNNs) and graph neural networks (GNNs)/multi-layer perceptrons (MLPs). Moreover, graph convolutions become less scalable as the number of vertices in the mesh grows to accommodate complex shapes, and often fail to learn diffeomorphic mappings to produce genus-zero regular meshes [10]. Third , using a coarse mesh template incurs difficulty of learning large deformations for highly folded cortical regions and may lead to non-smooth deformation and undesirable artifacts. Last but not least , the cortical thickness estimation is neglected in all existing DL-based CSR methods (i.e., needs a separate step to compute cortical attributes as in conventional pipelines [17, 56]). It may serve as an anatomical constraint to couple the inner and outer cortical surfaces, help generate cortical surfaces with topological correctness, and facilitate quantitative analysis of the cortical thickness.

In order to robustly reconstruct the cortical surfaces with topological correctness, we develop a DL-based approach to simultaneously reconstruct both the inner and outer cortical surfaces and estimate the cortical thickness by optimizing and deforming an initialized midthickness surface. First , our method explicitly couples the inner and outer surfaces by jointly learning three diffeomorphic flows to optimize the initialization midthickness surface to lie halfway between the inner and outer cortical surfaces and deform it to inner and outer cortical surfaces, respectively. Second , instead of designing a complex mixed architecture of CNNs and GNNs/MLPs, our method employs a single model of 3D CNNs to predict the diffeomorphic flows from a multi-channel input, consisting of a 3D brain MRI, a ribbon segmentation map that encodes structural information of the cerebral cortex, and a signed distance function that implicitly encodes the initialization surface. Third , our method calculates the diffeomorphic deformation trajectories in a continues coordinate space rather than on a 3D voxelwise grid, achieving higher sub-voxel accuracy from fine-grained velocity fields while maintaining reasonable computational efficiency even as the number of mesh vertices increases. Fourth , we devise an efficient and reliable approach to initialize a fine midthickness surface from a cortical ribbon segmentation result, followed by topology correction to ensure genus zero. Finally , a vertex-wise thickness estimation can be obtained by tracing the geodesic trajectory of each vertex during the mesh deformation process. In summary , our new DL framework differs from the existing DL-based CSR approaches in its coupled reconstruction of multiple surfaces and the simultaneous cortical thickness estimation, which facilitates robust reconstruction of the cortical surfaces with spherical topology. Ablation studies and comparison experiments on two large public datasets (ADNI [22] and OASIS [33]) have demonstrated that our method attains superior performance over the state-of-the-art methods [10, 13, 26, 30, 31, 47].

2 Related Works

2.1 Learning-based Cortical Surface Reconstruction

Recent years have witnessed a surge of interest in geometric DL-based methods for general computer vision tasks [15, 29, 49, 52, 54, 57, 58]) and biomedical object reconstruction [24, 25, 37, 38, 55].

However, their applications to biomedical tasks are limited to organs with simple shapes such as the liver and heart. The cerebral cortex, on the other hand, is a highly folded, thin structure with a significantly complex shape, necessitating more advanced approaches.

Table 1 summarizes two categories of CSR methods, i.e., implicit and explicit surface reconstruction. The implicit surface reconstruction methods typically learn a function that maps 3D coordinates to a continuous implicit representation of the surface, such as signed distance function [39] and occupancy field [34]. Compared to well-established pipelines [17], these methods have significantly improved inference efficiency (reducing time cost from hours to minutes). Despite their ability to generate surfaces at any desired resolutions [13, 18, 19, 41], these methods often rely on a marching cubes algorithm [27] to obtain a triangular mesh and post-processing topology correction methods [7] for ensuring correct spherical topology of the reconstructed surfaces.

The explicit surface reconstruction methods typically learn a mapping from an initialization mesh to a target surface and the mapping can be modeled as a deformation model [52, 53, 55]. Particularly, Pixel2Mesh [52] utilizes GNNs to learn vertex-wise deformations of an ellipsoid and increase the mesh resolution in a coarse-to-fine manner. Such a strategy has also been adopted in V oxel2Mesh [55] to reconstruct simple human organ surfaces (e.g., liver, hippocampus) from CT/MR images. Similarly, PialNN [31] learns to deform an input inner cortical (i.e., WM) surface to a target outer (pial) surface by a sequence of deformation blocks; TopoFit [20] warps a topologically-correct template surface to fit the WM surface by a series of graph convolutional blocks; and Vox2Cortex [10] deforms a brain template surface to target cortical surfaces by leveraging combined CNNs and GNNs. Another line of research leverages ODE to parameterize the deformation of vertices as diffeomorphic flows, such as CoticalFlow methods [26, 47] and CortexODE [30]. However, all these methods may produce intersecting inner and outer cortical surfaces because they predict the inner and outer cortical surfaces either using separate DL models or using a joint DL model without explicit constraints to penalize the generation of intersecting surfaces.

2.2 Diffeomorphic Deformation

A diffeomorphism is theoretically differentiable and invertible and guarantees smooth and one-to-one mapping [46]. Diffeomorphic deformation has been widely used in image registration [2, 4, 5, 35, 51]. It can be generated from a velocity field v by integrating an ordinary differential equation (ODE) [1],

where ϕ ( x , 0) = x . The velocity field can be stationary [4] or time-varying [8]. Standard numerical integration techniques, such as the Euler method and the Runge-Kutta method [11], can be used to perform the integration. CoticalFlow methods [26, 47] and CortexODE [30] parameterize the ODE by a neural network [12]. CoticalFlow requires the deformation field to be Lipschitz at each time step to ensure a bijective mapping with Lipschitz inverse and derives a stability condition for the numeric approximation of ϕ . As a chain of deformation modules is trained in sequential stages to predict a series of deformation fields that deform the initial mesh template to the target surface in a coarseto-fine manner, there is a risk of generating self-intersections due to the low-resolution template in the coarsest level model and the whole process is prone to error accumulation in multiple steps. CortexODE [30] is built upon NODE [12] and operates on a topologically-corrected high-resolution initialization mesh. The sufficient condition of diffeomorphism can be satisfied if the deformation network is Lipschitz continuous and a sufficiently small step size is used in numerical approximation. Most learning-based image registration approaches [6, 28, 36] adopt stationary velocity field (SVF), and the integral (displacement field) is computed on a voxel-grid using the scaling and squaring method [3], yielding comparable registration accuracy to conventional methods while significantly improving efficiency. Our work lies at the intersection of these methods in that our method uses CNNs to parameterize multiple SVFs based on a multi-channel input and train the SVFs jointly to optimize cortical surfaces under topology-preserving and inverse-consistent transformation regularizations.

3 Methodology

As illustrated in Fig. 1, our framework consists of two parts: a pipeline that estimates a midthickness surface, represented both explicitly as a 3D mesh and implicitly as a 3D distance map (Sect. 3.1), and

Raw Image

Signed Distance

Transformation

Segmentation

Topology Check

Surf. Extraction

Midthickness

Surface

FCN

VF1

VF2

VF3

DDM

Initial Surface

Mid surf.

DDM

DDM

𝑳

𝒄𝒉𝑮

𝑳

𝒄𝒉𝑾

DDM

DDM

WM surf.

GM surf.

𝑳

3D FCN

DDM

Deformation

Module

Data Flow

VF

Velocity Field

𝑆

!

𝑆

"

𝑆

#

𝑆

$

𝑆

"

%

𝑆

"

%%

(a) Surface initialization

(b) Cortical surface reconstruction network

an end-to-end fully convolutional network (FCN) that reconstructs multiple surfaces and estimate cortical thickness simultaneously (Sect. 3.2). Loss functions are presented in (Sect. 3.3).

3.1 Midthickness Surface Initialization

The human cerebral cortex is a 2D sheet with an average thickness of ∼ 2 . 5 mm and has a highly folded geometry with peaks (i.e., gyri) and grooves (i.e., sulci) [17]. The existing CSR methods typically use a smoothed template surface estimated from a group of subjects [10, 47], or a WM surface [30, 31] to initialize the surface reconstruction. However, our preliminary experiment shows that the closer the initial surface is to its target surface, the higher the reconstruction accuracy is (see Fig. 2(a)). Thus, we propose to extract the midthickness layer which lies halfway between the inner and outer cortical surfaces as an initialization surface, which brings three advantages: (1) Deforming a surface from the midthickness surface reduces the learning difficulty and improves accuracy by avoiding learning 'large' deformations. It also strikes a balance between optimizing the inner and outer surfaces, making it less challenging. (2) The distance between the midthickness surface and the pial surface can help prevent topology errors in cases of deep sulci, as this layer still provides a clear separation between them. (3) Deforming the midthickness surface inward and outward to the inner and outer surfaces establishes a one-to-one mapping that explicitly encodes the correspondence between surfaces, facilitating coupled surface learning and improving CSR accuracy.

A key challenge is how to obtain a surface as close to the midthickness surface as possible. A straightforward method is to inflate the WM surface along the normal direction by a pre-determined distance [30]. However, this method cannot accurately estimate the midthickness surface in that the cortical thickness varies across the cortex and the normal is merely an approximation using neighboring faces. We propose a more accurate method by leveraging the cortex ribbon segmentation map (i.e., the filled interior area of WM and pial surfaces). Given an input brain MRI volume I ∈ R D 1 × D 2 × D 3 , we utilize a 3D U-Net [44] to generate a WM segmentation map M W ∈ R D 1 × D 2 × D 3 and a GM segmentation map M G ∈ R D 1 × D 2 × D 3 (see Fig. 1(a)). The network is trained on a largescale public neuroimaging dataset [22] by minimizing the cross-entropy loss between the prediction and ground truth which can be obtained using existing pipelines [17, 45]. Based on the predicted segmentation map M W , we generate a signed distance function (SDF), K W ∈ R D 1 × D 2 × D 3 , using a distance transform algorithm: d ( v i ) = SDF ( v i ) is the minimal Euclidean distance of voxel v i ∈ I to the boundary voxels. Voxels with values equal to zero represent the surface boundaries and voxels with negative or positive values encode their distances to the surface boundaries inward or outward, respectively. Similarly, we generate an SDF, K G , for the pial (gray matter) surface. A new SDF can be obtained by averaging the WM and GM SDFs: K M = ( K G + K W ) / 2 , whose 0-level defines the midthickness surface implicitly. To ensure the midthickness surface maintains a spherical topology, a fast topology check and correction algorithm [7, 30] is then applied to the implicit surface K M .

0

0.1

0.2

0.3

0.4

0.5

WM

MID

GM

ASSD (mm)

GM

WM

Grid

SVF

Velocity

at a grid

Trajectory

A vertex

Interpolation

𝐯

!

!

!"

!

!

!

"

Pial Surf.

Mid Surf.

!

"#$

!

"#$

′

(b)

(c)

Pial Surf.

Mid Surf.

!

!"

!

"#$

!

"#$

′′

!

!

!

"

𝐯

"

Vertex's trajectory

in

𝐯

!

and

𝐯

"

resp.

Velocity filed

going outward

Velocity filed

going inward

Init. Surf.

Position

(a)

Finally, the initialization midthickness surface is extracted by the marching cubes algorithm [27] from the 0-level of K M and parameterized by a triangular mesh S 0 .

3.2 Coupled Reconstruction of Cortical Surfaces

Based on the formulation in Eq. 1, an accurate parameterization of the SVFs is crucial to modeling the diffeomorphic trajectory of each vertex and reconstructing the cortical surfaces. In this section, we will show how our method achieves the goals.

Feature Extraction from a Mutli-channel Input of Brain MRI, Cortical Ribbon, and Cortical Surface. The existing DL-based CSR studies have demonstrated that it is critical to fuse both image features extracted from the MRIs using CNNs and geometry features extracted from surface meshes using GNNs/MLPs. However, besides employing two different network architectures, these methods typically learn the image and geometry features separately before fusion, which does not adequately utilize image texture and surface geometric information. To better model the SVFs, we propose to learn features from a multi-channel input consisting of a brain MRI, its cortical ribbon segmentation maps, and its midthickness surface represented as an SDF: I comb = I © M W ⊕ G © K M , where © is channel-wise concatenation and M W ⊕ G is a multiclass mask (BG = 0, WM = 0.5, GM = 1).

Such a feature learning procedure brings two key benefits: (1) Utilizing heterogeneous features enables mutual knowledge distillation. The brain MRI contains detailed texture and semantic information but may include noise and irrelevant regions far from the target surfaces. The cortical ribbon segmentation maps contain structural/semantic information about the cortical sheet and can act as an attention guide for extracting informative features around its boundaries. The SDF implicitly embeds the surface location information and relative relation between all voxels. Together, the multi-channel input provides richer and complementary information for our model to reconstruct the surfaces. (2) Only a single CNN is needed, which simplifies the model design and improves efficiency. Features can be extracted in a single forward pass for all coordinates. When scaling up the number of vertices in mesh, we can interpolate in the feature space efficiently.

Coupled Learning of Cortical Surfaces. The goal is to learn diffeomorphic deformations that deform the initialization midthickness surface S 0 ⊂ R 3 to its target WM and pial surfaces, S W and S G . Taking into account the discrepancy between the initialization S 0 and the true midthickness surfaces S M , our method also learns a diffeomorphic deformation to optimize the initialization midthickness surface. In total, our method learns a function to model three diffeomorphic deformations f θ ( I comb , S 0 ) = ( ϕ M , ϕ W , ϕ G ) , using an FCN and several diffeomorphic deformation modules (DDMs). Specifically, the FCN has a similar architecture as U-Net [44], consisting of a 5-level hierarchical encoder-decoder with skip connections as shown in Fig. 1(b) (see Supplementary Materials for details). By taking the multi-channel input, the FCN learns to estimate three dense SVFs jointly, denoted by v M , v W , and v G . We then use v M to compute ϕ M that deforms S 0 to the true midthickness surface S M , v W to compute ϕ W that deforms S M inward to the WM surface S W , and v G to compute ϕ G that deforms S M outward to the pial surface S G . By doing so, we establish a one-to-one mapping across S W , S M , and S G .

However, since ϕ W and ϕ G are computed using different SVFs, they may cause non-invertible transformation around the midthickness surface. To address this issue, we utilize the property of diffeomorphic mapping to compute a symmetric deformation trajectory of each vertex and devise a

symmetric cycle function L cyc (Eq. 2) for training. Fig. 2(b) illustrates the deformation of a vertex p Mid outward to p GM using ϕ G ( v G ) followed by the deformation inward to p ′ Mid using ϕ W ( v W ), which should be as close to p Mid as possible. Similarly, p ′′ Mid should also be as close to p Mid as possible when deformed inward by ϕ W followed by the outward deformation by ϕ G . The symmetric cycle loss is formulated as:

where p ϕ b ◦ ϕ a represents deforming a vertex p using velocity fields v a and v b sequentially. It ensures simultaneous alignment of p ϕ W ◦ ϕ G ≃ p ≃ p ϕ G ◦ ϕ W for each vertex p ∈ S M . Fig. 3 demonstrates the effectiveness of L cyc in the coupled reconstruction of multiple surfaces in the network optimization. Moreover, we can trace the geodesic trajectory of each vertex during the mesh deformation process for estimating vertex-wise cortical thickness (Fig. 2(b)).

Trajectory from

𝑺

𝑴𝒊𝒅

to

𝑺

𝑮𝑴

Trajectory from

𝑺

𝑴𝒊𝒅

to

𝑺

𝑾𝑴

w/o

𝐿

!"!

w/

𝐿

!"!

Diffeomorphic Deformation Module (DDM). To numerically solve the ODE in Eq. 1, scaling and squaring (SS) method [3] can be applied on the image grid, followed by trilinear interpolation (Lint( · )) for vertices in continuous coordinates. However, there are three limitations: (1) Interpolation cannot guarantee the invertibility of diffeomorphic mapping; (2) Numerical errors are amplified in the two sequential steps of SS+Lint( · ); (3) Computation on grid voxels including no vertices is unnecessary. Hence, we utilize the DDM to directly compute the vertex-wise integral. As shown in Fig. 2(c), we obtain the velocity vector for a vertex with coordinate x by interpolating its neighboring velocity vectors ( := - → v N ( x ) ), i.e., - → v x = Lint ( - → v N ( x ) ) . The vertex then moves to a new coordinate - → v x · 1 T , where T is the total time steps. We can obtain overall deformation using this procedure in T steps.

3.3 Loss Functions

We design multiple loss functions to optimize the geometric precision of the reconstructed surfaces and regularize the SVFs for diffeomorphic deformation.

Mesh loss . It aims to minimize distances of the vertices between the predicted surface meshes S W (and S G ) and their corresponding ground truth (GT) meshes S ∗ by the bidirectional Chamfer distance [26]:

where p and p ∗ are the coordinates of vertices on meshes. We can compute L chG analogously. The mesh loss is L ch = L chW + L chG .

Trajectory loss. Starting from the midthickness surface, the trajectory length of the vertex moving to the WM and the pial surfaces should be equal. We propose to compute the mean square difference of the vertex's trajectories:

where L Mid → GM ( p ) = ∑ T t =0 ∆ ϕ G,t ( p ) is the accumulated Euclidean distance (i.e., trajectory length) of T steps of deformation. This term encourages the midthickness surface can be deformed to the inner and outer cortical surfaces with the same deformation path length. In other words, the midthickness surface should lie halfway between the inner and outer cortical surfaces.

Symmetric cycle loss. We formulate it as Eq. 2 to encourage ϕ W and ϕ G to be invertible.

Symmetric similarity loss. To optimize the midthickness surface to lie halfway between the inner and outer cortical surfaces, a magnitude difference constraint is adopted directly on the SVFs:

where v W represents reverse-directional v W .

Normal consistency loss. We also incorporate a normal consistency regularization term to promote robust learning of the surfaces and ensure their smoothness:

where e is an edge, f 0 and f 1 are e 's two neighboring faces with their unit normals n f 0 and n f 1 .

In summary, we combine all the losses to jointly optimize our DL model: L = λ 1 L ch + λ 2 L dist + λ 3 L cyc + λ 4 L ss + λ 5 L nc , where { λ i } i =1 , ··· , 5 are weights to balance the loss terms. We empirically set λ i = 1 ( i = 1 , · · · , 4) and λ 5 = 0 . 001 .

4 Experiments

We evaluated our method for reconstructing both white-matter (WM) and pial surfaces on two largescale datasets, (ADNI) [22] and OASIS [33], and compared it with state-of-the-art (SOTA) DL-based CSR methods. We also tested its robustness and performed ablation analyses.

Datasets. The ADNI-1 [22] dataset consists of 817 subjects and we randomly split it into 654, 50, and 113 subjects for training, validation, and testing, respectively. The OASIS-1 [33] dataset consists of 413 subjects and we randomly split it into 330, 25, and 58 for training, validation, and testing, respectively. The models were trained on the training set until they reached a loss plateau on the validation set, after which their performance was evaluated on the test set. We followed pre-processing protocols in previous works [10, 13, 26, 30] for fair comparison. The T1-weighted MRI scans were aligned rigidly to the MNI152 template and clipped to the size of 192 × 224 × 192 at 1 mm 3 isotropic resolution. The pseudo ground-truth of ribbon segmentation and cortical surfaces were generated using FreeSurfer v7.2.0 [17]. The intensity values of MRI scans, ribbon segmentation maps, and SDFs were normalized to [0 , 1] and the coordinates of the vertices were normalized to [ -1 , 1] . The WM and GM in the cortical ribbon segmentation maps were assigned values of 0.5 and 1 respectively.

Implementation details. Our framework was implemented in PyTorch [40] and trained on an NVIDIA P100 GPU of 16 GB memory. The 3D U-Net [44] for ribbon segmentation was trained for 200 epochs using Adam [23] optimization and achieved an average Dice index of 0.96 on the testing set. The CSR model was trained for 400 epochs using Adam ( β 1 = 0 . 9 , β 2 = 0 . 999 , ϵ = 1 e -10 , learning rate 1 e -4 ) to optimize the midthickness surface and reconstruct the WM and pial surfaces for each hemisphere. The surface meshes had ∼ 130 K vertices.

Evaluation metrics. Weutilized three distance-based metrics to measure the CSR accuracy, including Chamfer distance (CD), average symmetric surface distance (ASSD), and Hausdorff distance (HD). In particular, CD measures the mean distance between two sets of vertices [16, 52]; ASSD and HD measure the average and maximum distances between two surfaces [13, 50]. They are computed bidirectionally over ∼ 130 K points uniformly sampled from the predicted and target surfaces. A lower distance indicates a better result. We used the 90th percentile instead of the maximum because HD is sensitive to outliers [21]. We also utilized the ratio of self-intersection faces (SIF) to measure surface quality [13, 30].

4.1 Comparison with Related Works

From the two categories of existing DL-based cortical surface reconstruction methods described in Section 1, we selected representative ones from each category for comparison. The experimental results are summarized in Table 2 and illustrated in Figure 4.

Main Results & Analysis. It is evident that our method achieved substantial improvement on both the WM and pial surface reconstruction over other approaches. Since DeepCSR [13] predicts an SDF-based implicit surface and requires post-processing to correct topology and extract a mesh, its results may contain no SIFs but were less accurate compared with the explicit CSR methods. Starting from the WM surface, PialNN [31] can achieve sub-voxel accuracy but its SIF ratio was relatively high. Vox2Cortex [30] can generate multiple surfaces from different template meshes. It employs complex CNN and GNN models to model the deformation for each vertex but has no diffeomorphism guarantee. The promising results of [26, 30, 47] indicated that using neural networks to parameterize the ODE can facilitate the diffeomorphic deformation, yielding better CSR accuracy. Our method

achieved the overall best performance due to its explicit regularizations on the deformation trajectory of vertices and its better initialization surface. On the ADNI dataset, our method achieved ∼ 48 . 8% improvement in mean ASSD (of WM and pial surfaces) compared to the second best CortexODE (i.e., 0.104 mm v.s 0.203 mm ) with competitive self-intersection ratio (average ∼ 0.021 % ). On the OASIS dataset, our method achieved similar performance improvement. More quantitative results on the WM surface are reported in the Supplementary Materials. As shown in Fig. 4, the CSR results obtained by our method had uniformly smaller errors across the whole surfaces.

Pial surface

DeepCSR

PialNN

CorticalFlow++

vox2cortex

CortexODE

Ours

GT

WM surface

1

0.8

0.2

0.4

0

0.6

Error

map

Runtime Analysis. It took ∼ 0.8 s for our method to obtain a ribbon segmentation map and another ∼ 2 s for the topology correction and surface initialization. For the surface reconstruction, the inference time for simultaneously reconstructing three (i.e., WM, pial, and midthickness) surfaces was ∼ 1.5 s . In comparison, the SOTA explicit CSR method CortexODE [30] needed ∼ 2 s to reconstruct two surfaces sequentially . Overall, our method is computationally as efficient as the SOTA alternatives.

4.2 Ablation Studies

Input. Our method takes as input a multi-channel 3D images I comb and an initialization midthickness surface S 0 . We conducted two experiments to analyze the influence of the input components on the CSR performance (Table 3 Top). First , by gradually removing components from I comb , we observed a significant drop in accuracy, indicating the contribution of both the SDF and segmentation maps to the final results. Second , we investigated the impact of the fine structure of the initialization surface by applying Laplacian smoothing on S 0 to generate oversmoothed initialization surfaces. The results revealed that the accuracy of the model decreased with the increasing of smoothing steps which resulted in a coarser initialization surface S 0 .

Loss functions. We evaluated the contribution of different losses of our method to the surface reconstruction performance in terms of both accuracy (CD, ASSD, HD) and topological correctness (SIF), with the results summarized in Table 3 Middle. Through the ablation studies, we observed

that each component played its own role in a complementary way. The first row (referred to as S0) corresponds to our complete network setting, while the last row (S4) represents using Chamfer distance alone. The results of setting S4 indicated that the model generated surfaces well matched to the ground truth data at the cost of high topological errors, particularly in highly curved regions, reflected by the results that the SIF ratio was worse on the pial surface than on the WM surface. Enforcing equality of the trajectories from the midthickness surface to the WM and pial surfaces (S3, L dist ) helped optimize the midthickness surface, thereby preventing deformation in an arbitrary direction and reducing self-intersection. However, the geometric accuracy slightly decreased, which might be caused by the difficulty in accessing highly curved regions or deep sulci under such strong topology constraints. The proposed symmetric cycle loss (S2, L cyc ) promoted the invertibility of deformations, yielding a significant reduction of self-intersections on the meshes since our method jointly reconstructs both the inner and outer surfaces by deforming the midthickness surface inward and outward with two VFs. Such invertibility also facilitates accurate estimation of the cortical thickness from the trajectory, as illustrated by a sample vertex deformation trajectory in Fig. 2 obtained with settings of S2 and S3. Moreover, the inclusion of regularization terms on the smoothness of SVFs (S1, L ss ) and surfaces (S0, L nc ) contributed to enhancement in surface quality. Overall, our proposed method struck a balance between geometric accuracy and topology quality.

Deformation steps in DDM. Table 3 Bottom shows the evaluation results of different numbers of deformation steps ( T ) in DDM. As T increased, the performance first improved and then saturated, indicating that five steps were sufficient to deform the midthickness surface to the inner or outer surfaces.

4.3 Reproducibility

We carried out two experiments on two datasets: a paired ADNI1.5&3T dataset [22] consisting of 1.5T and 3T MRIs of the same subjects, and the Test-Retest dataset [32] comprising 40 MRIs collected within a short period for each of the 3 subjects. In these scenarios, the cortical surfaces of the same subject should be nearly identical. Following the experimental setup outlined in [10, 13, 30], we utilized the iterative closestpoint algorithm (ICP) to align image pairs and computed the geometric distance between surfaces. The

results for the left WM surfaces are presented in Table 4 (more in Supplementary Materials),

demonstrating that our method obtained superior reproducibility compared with FreeSurfer and was comparable to the SOTA DL methods.

4.4 Cortical Thickness

In contrast to the alternative methods that rely on the ICP algorithm for registering WM and pial surfaces prior to calculating the Euclidean distance [10], our proposed method directly provides vertex-wise cortical thickness estimation. To validate the cortical thickness estimation, we compared our method with FreeSurfer for estimating the cortical thickness. We identified 200 subjects from the ADNI-2GO [22] dataset (100 are diagnosed with Alzheimer's disease and 100 are normal controls) and computed the average cortical thickness across 35 cortical regions based on a surface parcellation provided by FreeSurfer [17]. Fig. 5 shows the correlation between ours and FreeSurfer's results, showcasing the effectiveness of our proposed framework in accurately capturing cortical thickness.

5 Limitations and Future Directions

Despite achieving improved CSR accuracy and a low SIF ratio, our method can be further improved by adopting post-processing methods and new loss functions in order to minimize the SIF ratio and improve surface quality. While focusing on cortical thickness estimation in this paper, we recognize the value of incorporating other cortical attributes like surface area and sulci depth into CSR and analysis tasks. These attributes could serve as complementary constraints, enhancing overall performance. It should be noted that further analysis is merited to thoroughly evaluate the proposed method on a large cohort of subjects (e.g., subjects in different stages of AD) although we have demonstrated the correlation between the estimated cortical thickness and that of FreeSurfer on a balanced dataset.

6 Conclusion

We introduce a new DL framework for cortical surface reconstruction by generating a midthickness surface to initialize a coupled reconstruction of both the WM and pial surfaces. Specifically, the midthickness surface is estimated from a 3D distance map from each MRI by generating a cortical ribbon segmentation map that encodes structural information of the cerebral cortex. The estimated midthickness surface is represented as a triangular mesh with spherical topology, and the mesh is optimized to lie at the center of the inner and outer cortical surfaces and deformed to the inner and outer cortical surfaces by three diffeomorphic flows that are learned jointly with CNNs optimized with a multi-channel input consisting of the brain MRI, the 3D distance map of midthckness surface, and the cortical ribbon segmentation map. Our proposed symmetric cycle loss helps learn diffeomorphic deformation and the numerical solution of DDM improves CSR accuracy and computation efficiency. Experiments on two large-scale neuroimage datasets have demonstrated the superior performance of our method. Moreover, our method generates an estimation of cortical thickness, facilitating statistical analyses of brain atrophy.

References

V. I. Arnold. Ordinary differential equations . Springer Science & Business Media, 1992.

1.50

1.75

2.00

2.25

2.50

2.75

3.00

3.25

2.0

2.5

3.0

3.5

4.0

Coefficients: m=1.16, b=0.00

Fitting Line

Cortical Regions

V. Arsigny. Processing data in lie groups: An algebraic approach. Application to non-linear registration and diffusion tensor MRI . PhD thesis, Citeseer, 2004.

V. Arsigny, O. Commowick, X. Pennec, and N. Ayache. A log-euclidean framework for statistics on diffeomorphisms. In International Conference on Medical Image Computing and Computer-Assisted , pages 924-931. Springer, 2006.

J. Ashburner. A fast diffeomorphic image registration algorithm. Neuroimage , 38(1):95-113, 2007.

B. B. Avants, C. L. Epstein, M. Grossman, and J. C. Gee. Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Medical Image Analysis , 12(1):26-41, 2008.

G. Balakrishnan, A. Zhao, M. R. Sabuncu, J. Guttag, and A. V. Dalca. Voxelmorph: a learning framework for deformable medical image registration. IEEE Transactions on Medical Imaging , 38(8):1788-1800, 2019.

P.-L. Bazin and D. L. Pham. Topology correction of segmented medical images using a fast marching algorithm. Computer Methods and Programs in Biomedicine , 88(2):182-190, 2007.

M. F. Beg, M. I. Miller, A. Trouvé, and L. Younes. Computing large deformation metric mappings via geodesic flows of diffeomorphisms. International Journal of Computer Vision , 61:139-157, 2005.

M. Bertoux, J. Lagarde, F. Corlier, L. Hamelin, J.-F. Mangin, O. Colliot, M. Chupin, M. N. Braskie, P. M. Thompson, M. Bottlaender, et al. Sulcal morphology in alzheimer's disease: an effective marker of diagnosis and cognition. Neurobiology of Aging , 84:41-49, 2019.

F. Bongratz, A.-M. Rickmann, S. Pölsterl, and C. Wachinger. Vox2Cortex: Fast explicit reconstruction of cortical surfaces from 3D MRI scans with geometric deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 20773-20783, 2022.

R. L. Burden, J. D. Faires, and A. M. Burden. Numerical Analysis . Cengage learning, 2015.

R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud. Neural ordinary differential equations. Advances in Neural Information Processing Systems , 31, 2018.

R. S. Cruz, L. Lebrat, P. Bourgeat, C. Fookes, J. Fripp, and O. Salvado. DeepCSR: A 3D deep learning approach for cortical surface reconstruction. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 806-815, 2021.

A. M. Dale, B. Fischl, and M. I. Sereno. Cortical surface-based analysis: I. segmentation and surface reconstruction. Neuroimage , 9(2):179-194, 1999.

G. Fahim, K. Amin, and S. Zarif. Single-view 3D reconstruction: A survey of deep learning methods. Computers & Graphics , 94:164-190, 2021.

H. Fan, H. Su, and L. J. Guibas. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 605-613, 2017.

B. Fischl. Freesurfer. Neuroimage , 62(2):774-781, 2012.

K. Gopinath, C. Desrosiers, and H. Lombaert. SegRecon: Learning joint brain surface reconstruction and segmentation from images. In International Conference on Medical Image Computing and Computer Assisted Intervention , pages 650-659. Springer, 2021.

Y. Hong, S. Ahmad, Y. Wu, S. Liu, and P.-T. Yap. Vox2Surf: Implicit surface reconstruction from volumetric data. In International Workshop on Machine Learning in Medical Imaging in Conjunction with MICCAI 2021 , pages 644-653. Springer, 2021.

A. Hoopes, J. E. Iglesias, B. Fischl, D. Greve, and A. V . Dalca. Topofit: Rapid reconstruction of topologically-correct cortical surfaces. In Medical Imaging with Deep Learning , 2022.

D. P. Huttenlocher, G. A. Klanderman, and W. J. Rucklidge. Comparing images using the Hausdorff distance. IEEE Transactions on Pattern Analysis and Machine Intelligence , 15(9):850863, 1993.

C. R. Jack Jr, M. A. Bernstein, N. C. Fox, P. Thompson, G. Alexander, D. Harvey, B. Borowski, P. J. Britson, J. L. Whitwell, C. Ward, et al. The alzheimer's disease neuroimaging initiative (ADNI): MRI methods. Journal of Magnetic Resonance Imaging: An Official Journal of the International Society for Magnetic Resonance in Medicine , 27(4):685-691, 2008.

D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In Y. Bengio and Y. LeCun, editors, International Conference on Learning Representations , 2015.

F. Kong and S. C. Shadden. Learning whole heart mesh generation from patient images for computational simulations. IEEE Transactions on Medical Imaging , 2022.

F. Kong, N. Wilson, and S. Shadden. A deep-learning approach for direct whole-heart mesh reconstruction. Medical Image Analysis , 74:102222, 2021.

L. Lebrat, R. Santa Cruz, F. de Gournay, D. Fu, P. Bourgeat, J. Fripp, C. Fookes, and O. Salvado. CorticalFlow: A diffeomorphic mesh transformer network for cortical surface reconstruction. Advances in Neural Information Processing Systems , 34, 2021.

T. Lewiner, H. Lopes, A. W. Vieira, and G. Tavares. Efficient implementation of marching cubes' cases with topological guarantees. Journal of Graphics Tools , 8(2):1-15, 2003.

H. Li, Y. Fan, and A. D. N. Initiative. MDReg-Net: Multi-resolution diffeomorphic image registration using fully convolutional networks with deep self-supervision. Human Brain Mapping , 43(7):2218-2231, 2022.

B. Ma, Z. Han, Y.-S. Liu, and M. Zwicker. Neural-pull: Learning signed distance function from point clouds by learning to pull space onto surface. In International Conference on Machine Learning , pages 7246-7257, 2021.

Q. Ma, L. Li, E. C. Robinson, B. Kainz, D. Rueckert, and A. Alansary. CortexODE: Learning cortical surface reconstruction by neural ODEs. IEEE Transactions on Medical Imaging , 2022.

Q. Ma, E. C. Robinson, B. Kainz, D. Rueckert, and A. Alansary. PialNN: A fast deep learning framework for cortical pial surface reconstruction. In International Workshop on Machine Learning in Clinical Neuroimaging , pages 73-81. Springer, 2021.

J. Maclaren, Z. Han, S. B. Vos, N. Fischbein, and R. Bammer. Reliability of brain volume measurements: a test-retest dataset. Scientific Data , 1(1):1-9, 2014.

D. S. Marcus, T. H. Wang, J. Parker, J. G. Csernansky, J. C. Morris, and R. L. Buckner. Open access series of imaging studies (OASIS): cross-sectional mri data in young, middle aged, nondemented, and demented older adults. Journal of Cognitive Neuroscience , 19(9):1498-1507, 2007.

L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger. Occupancy networks: Learning 3D reconstruction in function space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4460-4470, 2019.

M. Modat, D. M. Cash, P. Daga, G. P. Winston, J. S. Duncan, and S. Ourselin. Global image registration using a symmetric block-matching approach. Journal of Medical Imaging , 1(2):024003-024003, 2014.

T. C. Mok and A. Chung. Fast symmetric diffeomorphic image registration with convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4644-4653, 2020.

D. H. Pak, M. Liu, S. S. Ahn, A. Caballero, J. A. Onofrey, L. Liang, W. Sun, and J. S. Duncan. Weakly supervised deep learning for aortic valve finite element mesh generation from 3D CT images. In International Conference on Information Processing in Medical Imaging , pages 637-648. Springer, 2021.

D. H. Pak, M. Liu, T. Kim, L. Liang, R. McKay, W. Sun, and J. S. Duncan. Distortion energy for deep learning-based volumetric finite element mesh generation for aortic valves. In International Conference on Medical Image Computing and Computer Assisted Intervention , pages 485-494. Springer, 2021.

J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 165-174, 2019.

A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems , 32, 2019.

J. Ren, Q. Hu, W. Wang, W. Zhang, C. S. Hubbard, P. Zhang, N. An, Y. Zhou, L. Dahmani, D. Wang, et al. Fast cortical surface reconstruction from mri using deep learning. Brain Informatics , 9(1):1-16, 2022.

L. M. Rimol, R. Nesvåg, D. J. Hagler Jr, Ø. Bergmann, C. Fennema-Notestine, C. B. Hartberg, U. K. Haukvik, E. Lange, C. J. Pung, A. Server, et al. Cortical volume, surface area, and thickness in schizophrenia and bipolar disorder. Biological Psychiatry , 71(6):552-560, 2012.

J. M. Roe, D. Vidal-Piñeiro, Ø. Sørensen, A. M. Brandmaier, S. Düzel, H. A. Gonzalez, R. A. Kievit, E. Knights, S. Kühn, U. Lindenberger, et al. Asymmetric thinning of the cerebral cortex across the adult lifespan is accelerated in alzheimer's disease. Nature Communications , 12(1):721, 2021.

O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention , pages 234-241, 2015.

A. G. Roy, S. Conjeti, N. Navab, C. Wachinger, A. D. N. Initiative, et al. Quicknat: A fully convolutional network for quick and accurate segmentation of neuroanatomy. NeuroImage , 186:713-727, 2019.

D. Ruelle and D. Sullivan. Currents, flows and diffeomorphisms. Topology , 14(4):319-327, 1975.

R. Santa Cruz, L. Lebrat, D. Fu, P. Bourgeat, J. Fripp, C. Fookes, and O. Salvado. CorticalFlow++: Boosting cortical surface reconstruction accuracy, regularity, and interoperability. In International Conference on Medical Image Computing and Computer Assisted Intervention , pages 496-505. Springer, 2022.

D. W. Shattuck and R. M. Leahy. Brainsuite: an automated cortical surface identification tool. Medical Image Analysis , 6(2):129-142, 2002.

A. Sinha, A. Unmesh, Q. Huang, and K. Ramani. Surfnet: Generating 3d shape surfaces using deep residual networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 6040-6049, 2017.

A. A. Taha and A. Hanbury. Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Medical Imaging , 15(1):1-28, 2015.

T. Vercauteren, X. Pennec, A. Perchant, and N. Ayache. Diffeomorphic demons: Efficient non-parametric image registration. NeuroImage , 45(1):S61-S72, 2009.

N. Wang, Y. Zhang, Z. Li, Y. Fu, H. Yu, W. Liu, X. Xue, and Y.-G. Jiang. Pixel2Mesh: 3D mesh model generation via image guided deformation. IEEE Transactions on Pattern Analysis and Machine Intelligence , 43(10):3600-3613, 2020.

C. Wen, Y. Zhang, C. Cao, Z. Li, X. Xue, and Y. Fu. Pixel2Mesh++: 3D mesh generation and refinement from multi-view images. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(02):2166-2180, 2023.

X. Wen, J. Zhou, Y.-S. Liu, H. Su, Z. Dong, and Z. Han. 3d shape reconstruction from 2d images with disentangled attribute flow. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3803-3813, 2022.

U. Wickramasinghe, E. Remelli, G. Knott, and P. Fua. Voxel2Mesh: 3D mesh model generation from volumetric data. In International Conference on Medical Image Computing and ComputerAssisted Intervention , pages 299-308. Springer, 2020.

Z. Wu, F. Zhao, F. Wang, W. Lin, L. Wang, D. Shen, and G. Li. Surface-based analysis of the developing cerebral cortex. In Advances in Magnetic Resonance Technology and Applications , volume 2, pages 287-307. Elsevier, 2021.

H. Xie, H. Yao, S. Zhang, S. Zhou, and W. Sun. Pix2Vox++: Multi-scale context-aware 3d object reconstruction from single and multiple images. International Journal of Computer Vision , 128(12):2919-2935, 2020.

S. Yang, M. Xu, H. Xie, S. Perry, and J. Xia. Single-view 3d object reconstruction from shape priors in memory. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3152-3161, 2021.

H. Zheng, H. Li, and Y. Fan. Surf NN: Joint reconstruction of multiple cortical surfaces from magnetic resonance images. In International Symposium on Biomedical Imaging , pages 1-4, 2023.

Peer reviews:
comments: SUMMARY:
This paper is solving a solid practical problem, handling the cortical thickness estimation problem that was historically ignored. Moreover, it improves the quality of surface reconstruction. Authors observed that deforming an over-smoothed template results in lower accuracy if the template is relatively further away from the target. Therefore, reconstructing the midthickness layer could help improve the accuracy.

STRENGTHS:
(1) The problem to be solved is well-motivated, limitations of existing methods are analyzed.

(2) Figures are nicely made.

(3) Storytelling is good.

WEAKNESSES:
(1) Rather than integrating all the equations within the text, it would enhance clarity if you present them as numbered equations. Furthermore, providing explanations for each variable immediately after equations would help.

(2) Paragraphs are too long. Most subsections are single-paragraphed, and all contents are presented within that one paragraph.

QUESTIONS:
(1) Can you detail Figure1, so that it’s more informative? For example, the variables in the equations could appear in the figure at corresponding locations.

(2) Can you split into shorter paragraphs, where each paragraph conveys one observation / insight?

LIMITATIONS:
Yes
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method for reconstructing cortical surface (CSR) (including white matter (WM) and gray matter (GM)) from MRI images. The method first estimates the “midthickness” surface, which is the surface between WM and GM, and learn jointly diffeomorphic flows to deform it to WM and GM, respectively. The proposed model is then evaluated on 2 brain dataset including ADNI and OASIS and compare with other CSR methods.\n\nSTRENGTHS:\n- The submission is well-written, the authors adequately demonstrate their ideas in the paper. \n\n- In general, the proposed method provides a good quantitative result compared to other CSR methods.\n\nWEAKNESSES:\n- While I appreciate the reported improvement of the proposed method, I do not see a significant novel contribution of this work. Specifically, the idea of using Neural ODE to deform from the initial surface to target surface is widely explored in this task (CortexODE, CorticalFlow). The losses that the authors employed are also commonly applied in this task. Finally, the technical detail proposed in the paper does not seem to have a wide applicability in other areas.\n\nQUESTIONS:\n- In terms of qualitative results, I check the visualization of every method that the paper compares with, and none of them has equally distributed errors across the surface as illustrated in the paper (Fig 4). I wonder how the visualization is constructed to demonstrate your result. \n- The paper seems to be an improvement work of CortexODE. Instead of using the initial surface that is extracted from WM segmentation, the authors extract the mid surface, which is in between the inner surface and the outer surface as the initial surface. Regarding loss functions, I wonder why the SIF of the L-Pial surface when using only Chamfer loss is so bad (2.522), but SIF of the L-WM surface is still good (0.009). Furthermore, it seems to me that adding auxiliary losses other than Chamfer loss makes the model even worse (Table 3b). Specifically, when using only Chamfer loss (S4), the model outperforms the complete network setting loss (S0) in almost every metric. Instead of using only Chamfer loss, why do the authors propose to use multiple losses but get worse performance?\n\nLIMITATIONS:\nThe authors did not acknowledge any potential negative societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper focuses on the cortical surface-mesh reconstruction in Magnetic Resonance Imaging (MRI) data. The goal is to reconstruct both the inner and outer surfaces of the cortical region. The proposed method begins by estimating an intermediate midthickness layer, which is positioned halfway between the inner and outer surfaces. This midthickness layer serves as the initialized surface mesh for the reconstruction process.\n\nTo evaluate the effectiveness of the proposed method, the authors present results obtained from two neuroimage MRI datasets. These datasets likely consist of MRI scans of the brain, specifically targeting the cortical region. The authors use these datasets to showcase the performance of their surface reconstruction technique.\n\nBy employing their proposed method, the authors aim to demonstrate the ability to accurately reconstruct the cortical surfaces from MRI data. This can be valuable for various applications in neuroscience and neuroimaging research, such as brain mapping, cortical thickness analysis, and studying cortical abnormalities or diseases.\n\nSTRENGTHS:\nThe paper presents a significant advancement in cortical mesh reconstruction and thickness estimation. The key innovation lies in the incorporation of a constraint that ensures the model predicts both the inner and outer meshes while considering a plausible distance representing the cortical thickness between the white matter and gray matter. While previous studies have explored the use of ODE for cortical mesh reconstruction, the proposed method employs a distinct approach by leveraging this technique to obtain two mesh surfaces and incorporates several loss functions to optimize the results.\n\nOverall, the paper introduces a valuable contribution to the field by addressing the challenge of accurately reconstructing cortical meshes and estimating cortical thickness. The novel constraint and the unique application of ODE set this approach apart from previous works.\n\nWEAKNESSES:\nThe choice of spatial resolution for applying Marching Cubes on the Signed Distance Function (SDF) volume has a direct impact on the granularity of the resulting mesh, specifically in terms of the number of vertices generated. Increasing the number of vertices can potentially reduce the geometry reconstruction error, but this comes at the expense of higher computational requirements and increased memory consumption. Surprisingly, the paper lacks any explanation or analysis regarding this crucial aspect.\n\nIt would be beneficial for the paper to provide a detailed discussion or analysis concerning the trade-off between spatial resolution, mesh granularity, and the associated computational and memory costs. This would help readers understand the implications of different spatial resolutions on the quality of the reconstructed geometry. Additionally, examining the potential impact on downstream applications or comparing the performance across different resolutions could provide valuable insights for future research and practical implementations.\n\nQUESTIONS:\n- In your framework, after computing the average of WM and GM SDFs, the Marching Cubes algorithm is used for mesh reconstruction. What are the advantages of incorporating ODE?\n\n- What is the average number of vertices in the initialized mesh, and how does it affect the output metrics?\n\n- The caption of Figure 2 (a) does not adequately explain the graphs in the Figure. Please provide a clearer description in the text.\n\nLIMITATIONS:\nThe paper should thoroughly address the limitations associated with converting the segmentation map into an SDF volume, emphasizing how the resolution of the 3D segmentation volume affects accuracy. Additionally, discussing the potential implications or consequences of these limitations for downstream applications is crucial.\n\nTo provide a comprehensive analysis, it is essential to explicitly mention the failure cases and limitations of the proposed method. Discussing scenarios in which the method might not perform optimally or cases where it is not suitable would enhance the paper's credibility and assist readers in understanding the method's boundaries and potential weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a deep-learning-based approach for cortical surface reconstruction (CSR) with topology constraints to ensure the physical meaning of reconstructed surfaces. Experiments on public benchmarks show that the proposed approach outperforms current state of the art networks.\n\nSTRENGTHS:\nThe overall presentation is clear and easy to follow.\n\nIn cortical surface reconstruction, the reservation of the reconstructed surface geometry is a crucial and challenging part. This paper aims at tackling the problem by generating a mid-thickness surface with topology constraints, which is intuitive and well-motivated.\n\nThe experimental results seem to be encouraging and the proposed method outperforms SOTAs by a clear margin.\n\nWEAKNESSES:\nThe claimed contributions are over-stated, and several related works seem to be ignored and would potentially lead to misunderstanding for the readers.\n\"the cortical thickness estimation is neglected in all existing DL-based CSR methods\" is simply not true. There are plenty of related methods on cortical surface reconstruction/prediction/parcellation that consider cortical attributes (cortical thickness, surface area, cortical folding, diffusivity, structural and functional connectivity, etc) in their frameworks [1,2]. In [3], cortical thickness was already considered as a constraint during the network's supervised learning process, and their network was also designed to predict *both* the surfaces (inner and outer) and cortical thickness at the same time.\nSince all these works are closely related, the authors should at least be aware of and mention or discuss the relations and differences between these methods and their proposed one. \n\n[1] Wu et al. Surface-based analysis of the developing cerebral cortex. Advances in Magnetic Resonance Technology and Applications, 2021.\n[2] Zhao et al. Spherical Deformable U-Net: Application to Cortical Surface Parcellation and Development Prediction. IEEE Transactions on Medical Imaging, 2021.\n[3] Liu et al. Deep modeling of growth trajectories for longitudinal prediction of missing infant cortical surfaces. International Conference on Information Processing in Medical Imaging (IPMI), 2019.\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nThe authors did not discuss their method's limitations, or the potential future directions.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed a method for cortical surface reconstruction by deforming a middle layer inwards and outwards to be the gray matter and white matter surface respectively. First, segmentation network and topology correction are used to extract the initial surface. Then, three velocity fields are predicted to deform the initial surface to be the final white matter and gray matter. The experiments demonstrate the proposed method and the design.\n\nSTRENGTHS:\n1. The idea of deforming the middle surface to generate white matter and gray matter is interesting. \n2. The experiments are sound and demonstrating, including the comparison between the SOTA methods and the ablation study for the components.\n\nWEAKNESSES:\nThis paper seems to overstate its contribution. First, it claims that it devises an efficient and reliable approach for initialization. However, the initialization step is basically following CortexODE, where the CortexODE predicts the white segmentation, and initializes the WM surface, this paper initializes the midthickness surface. The pipeline of initialization is the same as proposed by CortexODE. Second, the diffeomorphic deformation. This paper claims that they design a new numerical computation strategy to calculate the diffeomorphic deformation trajectory, where the deformation is the accumulated velocity along the several time steps. The CortexODE also utilizes explicit fixed-step methods for ODE discretization instead of the scaling and squaring methods. \n\nThe idea of utilizing the middle state to generate the final result is also not new, a similar approach can be found in [1]. \n\n[1] Mok, Tony CW, and Albert Chung. \"Fast symmetric diffeomorphic image registration with convolutional neural networks\n\nQUESTIONS:\nFigure 1 is confusing to me, For the cyclic loss on the GM as the example, why the forward and backward deformation is from the same velocity field? In Eq. 2, they should use different velocities.\n\nLIMITATIONS:\nPlease try to solve the questions and weakness above.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper develops a new deep-learning-based framework to jointly reconstruct the inner, outer, and their in-between (midthickness) surfaces and estimate cortical thickness. The proposed method has been evaluated on two neuroimaging datasets and shown superior cortical surface reconstruction performance. This work is potentially interesting to the computational neuroscience community and valuable for brain study.\n\nSTRENGTHS:\nThe authors propose a novel deep-learning-based framework to robustly reconstruct the cortical surfaces with topological correctness. Different from existing methods, it explicitly couples the inner and outer surfaces by jointly learning three diffeomorphic flows. Instead of using mixed structures of CNN and GNNs/MLP, this work utilizes a single 3D CNN to leverage 3D brain MRI, a ribbon segmentation map, and a signed distance function altogether. Also, the diffeomorphic deformation trajectory is calculated by a new numerical computation strategy. Furthermore, the initialization of fine midthickness surfaces is effective, and thickness estimation is performed together with cortical reconstruction. Overall, this paper is well-written and easy to follow. The proposed method is novel and can potentially alleviate several limitations of existing methods. The experiments also demonstrate the effectiveness of the proposed method on two neuroimaging datasets.\n\nWEAKNESSES:\nThe demographics for the two testing datasets are not illustrated, and experiments can be performed on more populations. \nI am confused about the effectiveness of different losses in Table 3.\n\nQUESTIONS:\nSee limitations.\n\nLIMITATIONS:\nExperiments can be performed on more populations (e.g., babies) to evaluate the effectiveness. More explanations are needed to demonstrate the contributions of different losses in the ablation study (Table 3).", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-24"}