Repository: nerfstudio-project/nerfstudio

Issue:
Specify target types from command-line (configs)
Ideas from @brentyi 

In terms of choosing between datasets if we assume every dataset type needs different fields seems like there are two options:

(1) Just storing every possible parameter, and adding a field for selecting between the config types

This looks kind of dumb, but it's at least reasonably clear what's happening?
```
@dataclasses.dataclass
class VanillaDataloaderConfig:
    # The chosen dataset config.
    selected_dataset_type: Literal["blender", "friends", "mipnerf360", "record3d"] = "blender"

    # Configs for each dataset type. Only one will be used.
    dataset_config: Dict[str, DatasetConfig] = {
        "blender": BlenderDatasetConfig(),
        "friends": FriendsDatasetConfig(),
        "mipnerf360": MipNerf360DatasetConfig(),
        "record3d": Record3DDatasetConfig(),
    }
    ...
```
and then you could access the chosen dataset config in code via `dataloader_config.dataset_config[dataloader_config.selected_dataset_type]` ; I haven't checked, but I think this should all work in the current version of dcargs

(2) the less redundant alternative is creating subcommands by annotating `dataset_config` with a Union. this might be cleaner in the code, but worried subcommands might be slightly more confusing in the CLI?
Needs a bit of work on my end, but the annotation change would look roughly like:

```
class VanillaDataloaderConfig(InstantiateConfig):
     """Configuration for dataloader instantiation"""
     _target: Type = VanillaDataloader
-    train_dataset: DatasetConfig = BlenderDatasetConfig()
+    train_dataset: Union[
+        BlenderDatasetConfig,
+        FriendsDatasetConfig,
+        MipNerf360DatasetConfig,
+        Record3DDatasetConfig,
+    ] = BlenderDatasetConfig()
```

and then usage might look like:
`python scripts/run_train.py {non-dataset arguments} blender-dataset {dataset arguments}`

For option (2), if you do `python scripts/run_train.py --help`  all of the dataset config parameters will disappear. instead, there'll be a list of subcommands at the bottom of the helptext

![image](https://user-images.githubusercontent.com/3310961/189449970-5d07a55c-07d7-4610-9e66-daac7ab726f5.png)





Discussion:
If anybody has other ideas for what the perfect interface would like like here we can also try to make it happen!!

Between the two options above I'm now slightly biased toward (2) though:

```
python scripts/run_train.py instant-ngp {ngp args} blender-dataset {dataset args}
```
seems increasingly reasonable, and I think this pattern is easier to make consistent with the dataset viewing script for example:
```
python scripts/viewer/view_dataset.py blender-dataset {dataset args}
```

Another option is to just support one super-great-dataparser type, and provide some utilities for translating between it and/the common ones that others use? :shrug: 
Thanks @brentyi! Just trying to understand, is "blender-dataset" actually e.g., "pipeline.datamanager.train-dataparser:blender-data-parser-config" in practice?
Ah! Yeah, the API for doing so wasn't finalized yet when I sent the original Slack messages, but there's now a [`dcargs.conf.subcommand()`](https://brentyi.github.io/dcargs/api/dcargs/conf/#dcargs.conf.subcommand) you can use for tweaking this.

So the before/after would be from:

```python
DataParserUnion = Union[
    BlenderDataParserConfig,
    FriendsDataParserConfig,
]
```

To:
```python
# (1)

from typing_extensions import Annotated  # PEP 593 backport for Python<3.9

from dcargs.conf import subcommand

DataParserUnion = Union[
    Annotated[BlenderDataParserConfig, subcommand("blender-data", prefix_name=False)],
    Annotated[FriendsDataParserConfig, subcommand("friends-data", prefix_name=False)],
]
```

There are also options for constructing the same union type with less repetition; one example is that this could actually be handled with the same logic as base configs. I think this is nice for making things consistent:

```python
# (2) -- this is what the base config code does!

# pyright: strictDictionaryInference=true
# ^note: this is a file-level control
dataparser_base_configs = {
    "blender-data": BlenderDataParser(),
    "friends-data": FriendsDataParser(),
}
DataParserUnion = dcargs.extras.subcommand_type_from_defaults(
    dataparser_base_configs, prefix_names=False
)

def main(data: DataParserUnion) -> None:
    reveal_type(data)  # Should be correct
```

Or directly when classes are defined?

```python
# (3)

# this could go in nerfactory/configs/utils.py?
def dcargs_subcommand_name(
    name: str, prefix_name: bool = False
) -> Callable[[Type[T]], Type[T]]:
    """Annotate a class with the subcommand name used to select it when the class is
    used in a Union[] type that's passed to dcargs.

    Args:
        name: Subcommand name.
        prefix_name: Whether or not to include the standard dcargs nesting prefixes.

    Returns:
        A class decorator.
    """

    def inner(cls: Type[T]) -> Type[T]:
        return Annotated.__class_getitem__(  # type: ignore
            (
                cls,
                dcargs.conf.subcommand(name, prefix_name=prefix_name),
            )
        )

    return inner


@dcargs_subcommand_name("blender-data")
@dataclasses.dataclass
class BlenderDataParserConfig:
    ...


@dcargs_subcommand_name("friends-data")
@dataclasses.dataclass
class FriendsDataParserConfig:
    ...


DataParserUnion = Union[
    BlenderDataParserConfig,
    FriendsDataParserConfig,
]
```
I think I like (2) the most since it's fairly intuitive if one were to add a dataparser.
- (1) is also good, but (2) is seems like the cleaner & consistent option.
- (3) is neat but probably harder to figure out and easy to have a name conflict.

Resolving patch:
diff --git a/nerfactory/configs/base.py b/nerfactory/configs/base.py
--- a/nerfactory/configs/base.py
+++ b/nerfactory/configs/base.py
@@ -19,7 +19,7 @@
 from dataclasses import dataclass
 from datetime import datetime
 from pathlib import Path
-from typing import Any, Dict, List, Literal, Optional, Tuple, Type, Union
+from typing import Any, Dict, List, Literal, Optional, Tuple, Type
 
 import dcargs
 import torch
@@ -308,6 +308,21 @@ class Record3DDataParserConfig(DataParserConfig):
     more, images will be sampled approximately evenly. Defaults to 150."""
 
 
+AnnotatedDataParserUnion = dcargs.extras.subcommand_type_from_defaults(
+    {
+        "nerfactory-data": NerfactoryDataParserConfig(),
+        "blender-data": BlenderDataParserConfig(),
+        "friends-data": FriendsDataParserConfig(),
+        "mipnerf-360-data": MipNerf360DataParserConfig(),
+        "instant-ngp-data": InstantNGPDataParserConfig(),
+        "record3d-data": Record3DDataParserConfig(),
+    },
+    prefix_names=False,
+)
+"""Union over possible dataparser types, annotated with metadata for dcargs. This is the
+same as the vanilla union, but results in shorter subcommand names."""
+
+
 @dataclass
 class VanillaDataManagerConfig(InstantiateConfig):
     """Configuration for data manager instantiation; DataManager is in charge of keeping the train/eval dataparsers;
@@ -321,14 +336,7 @@ class VanillaDataManagerConfig(InstantiateConfig):
 
     _target: Type = VanillaDataManager
     """target class to instantiate"""
-    train_dataparser: Union[
-        NerfactoryDataParserConfig,
-        BlenderDataParserConfig,
-        FriendsDataParserConfig,
-        MipNerf360DataParserConfig,
-        InstantNGPDataParserConfig,
-        Record3DDataParserConfig,
-    ] = BlenderDataParserConfig()
+    train_dataparser: AnnotatedDataParserUnion = BlenderDataParserConfig()
     """specifies the dataparser used to unpack the data"""
     train_num_rays_per_batch: int = 1024
     """number of rays per batch to use per training iteration"""
diff --git a/scripts/run_train.py b/scripts/run_train.py
--- a/scripts/run_train.py
+++ b/scripts/run_train.py
@@ -215,4 +215,5 @@ def main(config: cfg.Config) -> None:
     # Create a subcommand for each base config.
     SubcommandType = dcargs.extras.subcommand_type_from_defaults(base_configs)
     instantiated_config = dcargs.cli(SubcommandType)
+
     main(config=instantiated_config)


Test patch: