Yes, you can just reference datasets that sit on your infrastructure or in the cloud.
For example, you can have your datasets on S3 and just reference the bucket.
run[“train_dataset”].track_files(“s3://datasets/train.csv”)
Neptune will save the following metadata about this dataset:
version (hash),
location (path),
size,
folder structure, and contents (files)
Neptune never uploads the dataset, just logs the metadata about it.
You can later compare datasets or group experiments by dataset version in the UI.