Add nifti support
State: closed

Add support for NIfTI.

supports #7804

This PR follows https://github.com/huggingface/datasets/pull/7325 very closely

I am a bit unsure what we need to add to the `document_dataset.mdx` and `document_load.mdx`. I should probably create a dataset on the hub first to create this guide instead of copy+pasting from PDF. 

Open todos:
 - [x] create nifti dataset on the hub
 - ~[ ] update `document_dataset.mdx` and `document_load.mdx`~
 
 EDIT:
 I tested with two datasets I created on the hub: 
  - https://huggingface.co/datasets/TobiasPitters/test-nifti-unzipped
  - https://huggingface.co/datasets/TobiasPitters/test-nifti
  
 for zipped (file extension `.nii.gz` and unzipped `.nii`) files and both seem to work fine. Also tested loading locally and that seems to work as well.
Here is the scriptsthat I ran against the hub:

```python
from pathlib import Path

from datasets import load_dataset
import nibabel as nib


dataset = load_dataset(
        "TobiasPitters/test-nifti-unzipped",
        split="test"  # Load as single Dataset, not DatasetDict
)

print("length dataset unzipped:", len(dataset))
for item in dataset:
    isinstance(item["nifti"], nib.nifti1.Nifti1Image)

dataset = load_dataset(
        "TobiasPitters/test-nifti",
        split="train"  # Load as single Dataset, not DatasetDict
)
print("length dataset zipped:", len(dataset))
for item in dataset:
    isinstance(item["nifti"], nib.nifti1.Nifti1Image)
```

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/datasets/pr_7815). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Btw I couldn't resist but share your PR with the community online on twitter already, I hope this is fine !

> Btw I couldn't resist but share your PR with the community online on twitter already, I hope this is fine !

Wow, that was quick! Thanks, already liked your comment, I appreciate it!

NIfTI support is out in `datasets==4.4.0` ! 🥳 

Btw do you know a good NIfTI vizualizer in HTML/JS or using python ? We could add something like `.to_html()` (or equivalent) to view data in a notebook and enable the Dataset Viewer on HF if it can be useful

cc @cfahlgren1 @georgiachanning for viz

Hi, I have a quick question while testing the new NIfTI support.

I cloned the latest `main` branch, installed it locally using `pip install -e .`, and ran the following:

```python
from datasets import load_dataset

dataset = load_dataset(
    "TobiasPitters/NIfTI-SIRF-exercises-geometry",
    split="train"
)
dataset[0]['nifti'].get_fdata()[0].shape
```

However, I’m getting the following error:

```
FileNotFoundError: No such file or no access: 'data/nifti/OBJECT_phantom_T2W_TSE_Sag_18_1.nii'
```

When I manually place the NIfTI file in that local path, it works fine.
But I assume the intended behavior is for the `.nii` file to be included in the dataset hosted on the Hub, so that `load_dataset()` automatically loads it without relying on a local file path.

Interestingly, after adding the `embed_storage` method, it started working properly.
Could you please confirm whether this is the expected behavior, or if my previous setup was missing something?

> ```python
> from datasets import load_dataset
> 
> dataset = load_dataset(
>     "TobiasPitters/NIfTI-SIRF-exercises-geometry",
>     split="train"
> )
> dataset[0]['nifti'].get_fdata()[0].shape
> ```

Thanks for the report, I can confirm this. It's a problem with the dataset. Can you try this:

```python
from datasets import load_dataset
import nibabel as nib

dataset = load_dataset(
        "TobiasPitters/test-nifti-unzipped",
        split="train"  # Load as single Dataset, not DatasetDict
)

print("length dataset:", len(dataset))
for item in dataset:
    assert isinstance(item["nifti"], nib.nifti1.Nifti1Image)
```
If your interested in `"TobiasPitters/NIfTI-SIRF-exercises-geometry"` I can give it a shot to reupload correctly, otherwise I'd take it down.

> NIfTI

I would suggest https://github.com/rii-mango/Papaya, just tested it and it looks quite good. How would it work to add that to the dataset-viewer?

And I assume you'd like to have the `to_html` feature on the `NifTI` class?

EDIT: do we have anything like this already for other features? Couldn't find anything. I mean we can do this in different ways, simply inlining papaya or building custom components (like e.g. SHAP is doing). If we decide for the latter, this means that we'll need to build js components in datasets, so we'll need a bundler, etc. but this provides the highest flexibility. If that's of interest, I can take a look into this.

Hi, thanks for your help earlier. I tested the dataset you shared (TobiasPitters/test-nifti-unzipped), and it works perfectly — all NIfTI files load correctly and get_fdata() returns valid arrays.

However, when I upload my own dataset to the Hugging Face Hub using my code, it doesn’t work properly. The NIfTI files seem not to decode correctly. can you check it?

```
train_dataset = Dataset.from_pandas(train_df)

def cast_dataset(dataset):
    dataset = dataset.cast_column("nifti", Nifti(decode=True))
    dataset = dataset.cast_column("label", ClassLabel(num_classes=10, names=[str(i) for i in range(10)]))

train_dataset = cast_dataset(train_dataset)
```

> Hi, thanks for your help earlier. I tested the dataset you shared (TobiasPitters/test-nifti-unzipped), and it works perfectly — all NIfTI files load correctly and get_fdata() returns valid arrays.
> 
> However, when I upload my own dataset to the Hugging Face Hub using my code, it doesn’t work properly. The NIfTI files seem not to decode correctly. can you check it?
> 
> ```
> train_dataset = Dataset.from_pandas(train_df)
> 
> def cast_dataset(dataset):
>     dataset = dataset.cast_column("nifti", Nifti(decode=True))
>     dataset = dataset.cast_column("label", ClassLabel(num_classes=10, names=[str(i) for i in range(10)]))
> 
> train_dataset = cast_dataset(train_dataset)
> ```

Are you using zipped Nifti files? It seems like there is an issue with that. I found that this creates problems, that in `decode_example` the path is something like `'gzip://T1.nii::/home/tobias/programming/github/datasets/nitest-balls1/NIFTI/T1.nii.gz'`, and then we go down the remote path which results in an `KeyError` since repo_id is not specified. The root cause for this is in the [DownloadManager.extract](https://github.com/huggingface/datasets/blob/main/src/datasets/download/download_manager.py#L297-L306) method, where we extract compressed files. 

@lhoestq : what do you suggest here? We could probably do something like this in the [`decode_example`](https://github.com/huggingface/datasets/blob/main/src/datasets/features/nifti.py#L144):
```python
if path.startswith("gzip:"):
    path = path.split("::")[-1]
```
Though I would need to test if this is actually OS agnostic.
    

I think the issue with gzip can be fixed using the same code as in Image() imo:
```diff
- try:
-     repo_id = string_to_dict(source_url, pattern)["repo_id"]
-     token = token_per_repo_id.get(repo_id)
- except ValueError:
-     token = None
+ source_url_fields = string_to_dict(source_url, pattern)
+ token = (
+     token_per_repo_id.get(source_url_fields["repo_id"]) if source_url_fields is not None else None
+ )
```


> Hi, thanks for your help earlier. I tested the dataset you shared (TobiasPitters/test-nifti-unzipped), and it works perfectly — all NIfTI files load correctly and get_fdata() returns valid arrays.
> 
> However, when I upload my own dataset to the Hugging Face Hub using my code, it doesn’t work properly. The NIfTI files seem not to decode correctly. can you check it?
> 
> ```
> train_dataset = Dataset.from_pandas(train_df)
> 
> def cast_dataset(dataset):
>     dataset = dataset.cast_column("nifti", Nifti(decode=True))
>     dataset = dataset.cast_column("label", ClassLabel(num_classes=10, names=[str(i) for i in range(10)]))
> 
> train_dataset = cast_dataset(train_dataset)
> ```

Can you pls try with [this branch](https://github.com/CloseChoice/datasets/tree/fix-embed-storage-nifti): 
```
pip install git+https://github.com/CloseChoice/datasets.git@fix-embed-storage-nifti
```
This should fix the existing problems with NifTI

I checked and it looks like the fix-embed-storage-nifti branch has already been merged into the main. And it worked fine. thanks.