hemants01/BrainTumorSegmentation

Model Card for Model ID

This model performs multi-class semantic segmentation of gliomas (brain tumors) on 2D axial MRI slices using four modalities (T1, T1CE, T2, FLAIR). The model is optimized for research use based on the BraTS 2020 challenge data.

Model Details

Model Description

This is a deep learning model for medical image segmentation. It processes a 4-channel 128 X 128 image patch (from a single axial slice across four MRI modalities) and outputs a pixel-wise mask classifying regions into four classes: Background, Necrotic Core/Non-Enhancing Tumor (NCR/NET), Peritumoral Edema (ED), and Enhancing Tumor (ET).

Model Sources [optional]

• Repository: hemants01/BrainTumorSegmentation
• Demo: hemants01/BrainTumorSegmentationDemo

Uses

The model's primary direct use is for brain tumor segmentation research, method comparison, and prototyping. It provides 2D segmentation masks when supplied with correctly preprocessed multi-modal MRI slices.

Out-of-Scope Use

Clinical Diagnosis or Treatment: This model must not be used for patient diagnosis, treatment planning, or any form of clinical decision-making. It is a research artifact.

Non-BraTS Data: Using the model on MRI data acquired with different protocols or without the required four co-registered modalities is likely to result in poor performance.

2D Segmentation: The model processes 2D slices independently. Its output must not be assumed to be a spatially smooth or coherent 3D segmentation mask without 3D post-processing.

Bias, Risks, and Limitations

Domain Specificity: Performance is tied to the characteristics of the multi-institutional BraTS dataset. Generalizing to different pathologies, scanners, or populations not represented in BraTS may lead to lower performance.

Modality Dependency: All four modalities (T1, T1CE, T2, FLAIR) are strictly required for inference.

Recommendations

Users (both direct and downstream) should be made aware that the model's output is an algorithmic prediction and must be validated by a domain expert (e.g., a neuroradiologist) if used in any clinical context (which is strongly discouraged).

How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

Training Details

Training Data

The model was trained on the BraTS 2020 dataset. This dataset comprises MRI scans of patients. Data Characteristics: Each subject includes four co-registered modalities (T1, T1CE, T2, FLAIR) as NIfTI files (.nii.gz) and skull-stripped. Annotations: Ground truth labels for tumor subregions (NCR/NET, ED, ET) were manually segmented by experts.

Training Procedure

Preprocessing [optional]

The training procedure relies on a multi-step preprocessing pipeline: 3D Volume Normalization: Z-score normalization applied per modality to the entire 3D volume. Slicing: Axial slices were extracted. Intensity Clipping: Values were clipped between the 2nd and 98th percentiles, then scaled to 0, 255. Resizing: Images were resized from 240 X 240 to 128 X 128 (bilinear interpolation). Labels/masks were resized with Nearest-neighbor interpolation (order=0) to the same image size. Global Normalization: Final per-tensor Z-score normalization applied to the 4-channel tensor. Augmentation: Extensive geometric and intensity augmentations were applied:

• Geometric :
• HorizontalFlip (50% probability)
• VerticalFlip (50% probability)
• Rotate limit=15 degrees (50% probability)
• ShiftScaleRotate shift_limit=0.05, scale_limit=0.05, rotate_limit=15 (50% probability)
• Intensity :
• RandomBrightnessContrast brightness_limit=0.2, contrast_limit=0.2 (50% probability)
• GaussNoise (20% probability)
• Custom CLAHE (A.Lambda) (20% probability): Applies Contrast Limited Adaptive Histogram Equalization (clip_limit=2.0, tile_grid_size=(8, 8)) independently to each of the four MRI channels.

Training Hyperparameters

• Training regime: Mixed Precision (FP16/bf16 via torch.amp.autocast)
• Loss Function: Tversky Loss (alpha=0.3, beta=0.7, ignore_index=0 (background))
• Optimizer: AdamW lr:1e-5
• Batch Size: 32
• Epochs: 1000 (with Early Stopping)

Evaluation

Testing Data, Factors & Metrics

Testing Data

A separate, held-out test set from the BraTS 2020 data (10%) was used.

Factors

Evaluation was performed across four classes (0-3), with aggregated scores often reported for the three clinically relevant tumor subregions:

(Enhancing Tumor (ET): Class 3

Tumor Core (TC): Classes 1 and 3 (NCR/NET + ET)

Whole Tumor (WT): Classes 1, 2, and 3 (NCR/NET + ED + ET)

Metrics

• Dice Score
• IoU Score
• Precision

Results

Dice Score

Subregion	Score
Whole Tumor (WT)	0.8941
Enhancing Tumor (ET)	0.7864
Tumor Core (TC)	0.7569

---

IOU Score

Subregion	Score
Whole Tumor (WT)	0.8187
Enhancing Tumor (ET)	0.7106
Tumor Core (TC)	0.6900

---

Precision

Subregion	Score
Whole Tumor (WT)	0.8834
Tumor Core (TC)	0.8502
Enhancing Tumor (ET)	0.8296

Model Architecture

The model is a highly specialized, hybrid network designed for medical image segmentation. It uses a modified U-Net structure that integrates a powerful encoder with a sophisticated global context bottleneck.

Component	Description	Rationale
Encoder	ResNeXt50_32x4d backbone, pre-trained on ImageNet. Initial Conv layer is adapted for 4 input channels (multimodality MRI).	Provides rich, deep, and hierarchically organized feature extraction.
Intermediate CNN	CNN blocks inserted after each ResNeXt layer block.	Further refines and regularizes deep feature maps before the next stage.
Transformer	Final encoder features are processed by Transformer Blocks with Positional Encoding.	Captures global, long-range dependencies across the feature map, mitigating the local view of pure CNNs.
ASPP	ASPP (Atrous Spatial Pyramid Pooling) module follows the Transformer.	Aggregates multi-scale features using dilated convolutions, essential for correctly segmenting tumor regions of various sizes.
Decoder	A standard upsampling path that reverses the encoder.	Restores the feature map resolution back to the original input size
Skip Connections	Attention Gates are used to connect encoder features to the decoder path.	Dynamically learns to weigh the importance of features in the skip connections, suppressing irrelevant background context and enhancing salient tumor features.
Final Layer	Convolutional layer.	Generates the final segmentation map with 4 output channels.

---

Compute Infrastructure

[More Information Needed]

Hardware

[More Information Needed]

Software

Operating System: Linux (e.g., Ubuntu, CentOS)

Core Framework: PyTorch (version 1.10 or later recommended)

Essential Libraries:

• torch
• numpy
• nibabel
• albumentations
• scikit-image
• skimage.transform.resize
• pandas
• tqdm
• matplotlib