Package: nobrainer
NobrainerNobraineris a deep learning framework for 3D image processing. It implements
several 3D convolutional models from recent literature, methods for loading and
augmenting volumetric data that can be used with any TensorFlow or Keras model,
losses and metrics for 3D data, and simple utilities for model training, evaluation,
prediction, and transfer learning.Nobraineralso provides pre-trained models for brain extraction, brain segmentation,
brain generation and other tasks. Please see theTrainedmodelsrepository for more information.TheNobrainerproject is supported by NIH RF1MH121885 and is distributed under
the Apache 2.0 license. It was started under the support of NIH R01 EB020470.Table of contentsImplementationsModelsDropout and additional layersLossesMetricsAugmentation methodsGuide Jupyter NotebooksInstallationContainerGPU supportCPU onlypipUsing pre-trained networksPredicting a brainmask for a T1-weighted brain scanGenerating a synthetic T1-weighted brain scanTransfer learningData augmentationRandom rigid transformationPackage layoutQuestions or issuesImplementationsModelsModelTypeApplicationHighresnet(source)supervisedsegmentation/classificationUnet(source)supervisedsegmentation/classificationVnet(source)supervisedsegmentation/classificationMeshnet(source)supervisedsegmentation/clssificationBayesian Meshnet(source)bayesian supervisedsegmentation/classificationBayesian Vnetbayesian supervisedsegmentation/classificationSemi_Bayesian Vnetsemi-bayesian supervisedsegmentation/classificationDCGANself supervisedgenerative modelProgressive GANself supervisedgenerative model3D Autoencoderself supervisedknowledge representation/dimensionality reduction3D Progressive Autoencoderself supervisedknowledge representation/dimensionality reduction3D SimSiam(source)self supervisedSiamese Representation LearningDropout and regularization layersBernouli dropout layer,Concrete dropout layer,Gaussian dropout,Group normalization layer,Costom padding layerLossesDice,Jaccard,Tversky,ELBO,Wasserstien,Gradient PenaltyMetricsDice,Generalized Dice,Jaccard,Hamming,TverskyAugmentation methodsSpatial TransformsCenter crop,Spatial Constant Padding,Random Crop,Resize,Random flip (left and right)Intensity TransformsAdd gaussian noise,Min-Max intensity scaling,Costom intensity scaling,Intensity masking,Contrast adjustmentAffine TransformAfifine transformation including rotation, translation, reflection.Guide Jupyter NotebooksPlease refer to the Jupyter notebooks in theguidedirectory to get
started withNobrainer.Try them outin Google Colaboratory!InstallationContainerWe recommend using the officialNobrainerDocker container, which includes all
of the dependencies necessary to use the framework. Please see the available images
onDockerHubGPU supportTheNobrainercontainers with GPU support use the Tensorflow jupyter GPU containers. Please
check the containers for the version of CUDA installed. Nvidia drivers are not included in
the container.$ docker pull neuronets/nobrainer:latest-gpu
$ singularity pull docker://neuronets/nobrainer:latest-gpuCPU onlyThis container can be used on all systems that have Docker or Singularity and
does not require special hardware. This container, however, should not be used
for model training (it will be very slow).$ docker pull neuronets/nobrainer:latest-cpu
$ singularity pull docker://neuronets/nobrainer:latest-cpupipNobrainercan also be installed with pip.$ pip install nobrainerUsing pre-trained networksPre-trained networks are available in theTrainedmodelsrepository. Prediction can be done on the command-line withnobrainer predictor in Python. Similarly, generation can be done on the command-line withnobrainer generateor in Python.Predicting a brainmask for a T1-weighted brain scanFigure: In the first column are T1-weighted brain scans, in the middle
are a trained model's predictions, and on the right are binarized FreeSurfer
segmentations. Despite being trained on binarized FreeSurfer segmentations,
the model outperforms FreeSurfer in the bottom scan, which exhibits motion
distortion. It took about three seconds for the model to predict each brainmask
using an NVIDIA GTX 1080Ti. It takes about 70 seconds on a recent CPU.In the following examples, we will use a 3D U-Net trained for brain extraction and
documented inTrainedmodels.In the base case, we run the T1w scan through the model for prediction.# Get sample T1w scan.wget-nchttps://dl.dropbox.com/s/g1vn5p3grifro4d/T1w.nii.gz
dockerrun--rm-v$PWD:/dataneuronets/nobrainer\predict\--model=/models/neuronets/brainy/0.1.0/brain-extraction-unet-128iso-model.h5\--verbose\/data/T1w.nii.gz\/data/brainmask.nii.gzFor binary segmentation where we expect one predicted region, as is the case with
brain extraction, we can reduce false positives by removing all predictions not
connected to the largest contiguous label.# Get sample T1w scan.wget-nchttps://dl.dropbox.com/s/g1vn5p3grifro4d/T1w.nii.gz
dockerrun--rm-v$PWD:/dataneuronets/nobrainer\predict\--model=/models/neuronets/brainy/0.1.0/brain-extraction-unet-128iso-model.h5\--largest-label\--verbose\/data/T1w.nii.gz\/data/brainmask-largestlabel.nii.gzBecause the network was trained on randomly rotated data, it should be agnostic
to orientation. Therefore, we can rotate the volume, predict on it, undo the
rotation in the prediction, and average the prediction with that from the original
volume. This can lead to a better overall prediction but will at least double the
processing time. To enable this, use the flag--rotate-and-predictinnobrainer predict.# Get sample T1w scan.wget-nchttps://dl.dropbox.com/s/g1vn5p3grifro4d/T1w.nii.gz
dockerrun--rm-v$PWD:/dataneuronets/nobrainer\predict\--model=/models/neuronets/brainy/0.1.0/brain-extraction-unet-128iso-model.h5\--rotate-and-predict\--verbose\/data/T1w.nii.gz\/data/brainmask-withrotation.nii.gzCombining the above, we can usually achieve the best brain extraction by using--rotate-and-predictin conjunction with--largest-label.# Get sample T1w scan.wget-nchttps://dl.dropbox.com/s/g1vn5p3grifro4d/T1w.nii.gz
dockerrun--rm-v$PWD:/dataneuronets/nobrainer\predict\--model=/models/neuronets/brainy/0.1.0/brain-extraction-unet-128iso-model.h5\--largest-label\--rotate-and-predict\--verbose\/data/T1w.nii.gz\/data/brainmask-maybebest.nii.gzGenerating a synthetic T1-weighted brain scanFigure: Progressive generation of T1-weighted brain MR scan starting
from a resolution of 32 to 256 (Left to Right: 323, 643,
1283, 2563). The brain scans are generated using the same
latents in all resolutions. It took about 6 milliseconds for the model to generate
the 2563brainscan using an NVIDIA TESLA V-100.In the following examples, we will use a Progressive Generative Adversarial Network
trained for brain image generation and documented inTrainedmodels.In the base case, we generate a T1w scan through the model for a given resolution.
We need to pass the directory containing the models(tf.SavedModel)created
while training the networks.dockerrun--rm-v$PWD:/dataneuronets/nobrainer\generate\--model=/models/neuronets/braingen/0.1.0\--output-shape=128128128\/data/generated.nii.gzWe can also generate multiple resolutions of the brain image using the same
latents to visualize the progression# Get sample T1w scan.dockerrun--rm-v$PWD:/dataneuronets/nobrainer\generate\--model=/models/neuronets/braingen/0.1.0\--multi-resolution\/data/generated.nii.gzIn the above example, the multi resolution images will be saved asgenerated_res_{resolution}.nii.gzTransfer learningThe pre-trained models can be used for transfer learning. To avoid forgetting
important information in the pre-trained model, you can apply regularization to
the kernel weights and also use a low learning rate. For more information, please
see theNobrainerguide notebook on transfer learning.As an example of transfer learning,@kaczmarjre-trained a brain extraction model to label meningiomas in 3D T1-weighted,
contrast-enhanced MR scans. The original model is publicly available and was
trained on 10,000 T1-weighted MR brain scans from healthy participants. These
were all research scans (i.e., non-clinical) and did not include any contrast
agents. The meningioma dataset, on the other hand, was composed of relatively
few scans, all of which were clinical and used gadolinium as a contrast agent.
You can observe the differences in contrast below.Despite the differences between the two datasets, transfer learning led to a much
better model than training from randomly-initialized weights. As evidence, please
see below violin plots of Dice coefficients on a validation set. In the left plot
are Dice coefficients of predictions obtained with the model trained from
randomly-initialized weights, and on the right are Dice coefficients of predictions
obtained with the transfer-learned model. In general, Dice coefficients are higher
on the right, and the variance of Dice scores is lower. Overall, the model on the
right is more accurate and more robust than the one on the left.Package layoutnobrainer.io: input/output methodsnobrainer.layers: custom layers, which conform to the Keras APInobrainer.losses: loss functions for volumetric segmentationnobrainer.metrics: metrics for volumetric segmentationnobrainer.models: pre-defined Keras modelsnobrainer.training: training utilities (supports training on single and multiple GPUs)nobrainer.transform: random rigid transformations for data augmentationnobrainer.volume:tf.data.Datasetcreation and data augmentation utilitiesCitationIf you use this package, pleaseciteit.Questions or issuesIf you have questions aboutNobraineror encounter any issues using the framework,
pleasesubmit a GitHub issue.