Package: scalingtheunet
Scaling the U-netRequirements to use this projectpoetrypython 3.9You can use many ways to get a specific python version. I recommend CondaConda installationOnce installed you need to install a python 3.9 interpreter:# Condaconda create -n sun_py39 python=3.9conda activate sun_py39Install poetry:# Poetry for Ubuntucurl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -How to use# clone projectgit clone https://gitlab.desy.de/ivo.matteo.baltruschat/scaling_the_u_net.gitcd scaling_the_u_netpoetry installRename.env.exampleto.env. SetPROJECT_DIRto your root path. If you wantwandbsupport, setWANDB_USERto your user nameRundvc checkout --summaryto restore raw and processed data from theshared_cache(/beegfs/desy/group/it/ReferenceData/dvc/shared_cache).Now your can simply runpython run_training.py experiment=baseline.Afterwards you can userun_testing.pyto evaluate your trained model on the test datarunpython run_testing.py experiment_dir=[your_experiment_name](replace [...] with your experiment subdir in the experiment folder)GuideHow To Get StartedFirst, you should probably get familiar withTensorflowNext, go throughHydra quick start guideandbasic Hydra tutorialHow it worksBy design, every run is initialized byrun_training.pyfile. All modules are dynamically instantiated from module paths specified in config. Example model config:filters:64# Channel output of first layer -> other depend on this onekernel_size:[3,3]# Kernel size for all Conv layersnum_layers:5# Number of u-net blocksregularization_factor_l1:-1# If -1, no regularization.regularization_factor_l2:-1# If -1, no regularization.layer_order:"CNAD"# Choose the oder of Conv,Norm,Act,Drop ["CADN", "CAND", "CDAN", "CDNA", "CNDA", "CNAD"]dropout_type:"spatial"# ["standard", "spatial"] Type of dropout used in a "CNAD" stackdropout_conv:-1# If -1, no dropout used in a "CNAD" stackuse_norm:"BatchNorm"# ["none", "BatchNorm", "GroupNorm"] Type of normalization used in a "CNAD" stackactivation:"mish"# ["relu", "leakyReLU", "mish"] Type of activation used in a "CNAD" stackdropout:-1# Dropout before the Conv-bock of latent space. If -1, no dropout used.output_activation:"linear"# ["linear", "softmax"] If "linear", model will provide logits and not prediction probabilitiesThis allows you to easily iterate over new models!Every time you create a new one, just specify its module path and parameters in appriopriate config file.The whole pipeline managing the instantiation logic is placed inscalingtheunet/executor/training.py.Main Project ConfigurationLocation:configs/config.yamlMain project config contains default training configuration.It determines how config is composed when simply executing commandpython run_training.py.
You can overwrite all parameters by cmd argumentspython run_training.py experiment_name=test123 model.filters=16Show main project configuration# specify here default training configuration# @package _global_# specify here default training configurationdefaults:-_self_-trainer:default.yaml-model:simple_unet.yaml-datamodule:mg_full.yaml-callbacks:default.yaml-logger:default.yaml#- mode: default.yaml-experiment:null# enable color logging-override hydra/hydra_logging:colorlog-override hydra/job_logging:colorlog# name of the run, accessed by loggers# allows for custom naming of the experimentname:???current_time:${now:%Y-%m-%d}_${now:%H-%M-%S}hydra:# sets output paths for all file logs to `logs/experiment/name'run:dir:experiments/${name}/${current_time}sweep:dir:experiments/${name}/${current_time}subdir:${hydra.job.num}output_subdir:"hydra_training"# path to original working directory# hydra hijacks working directory by changing it to the current log directory,# so it's useful to have this path as a special variable# https://hydra.cc/docs/next/tutorials/basic/running_your_app/working_directorywork_dir:${hydra:runtime.cwd}# path to folder with datadata_dir:${work_dir}/data/# mlflow pathmlflow_dir:null#${work_dir}/mlflow/# tensorboard pathtensorboard_dir:${work_dir}/tensorboard/# pretty print config at the start of the run using Rich libraryprint_config:True# pretty print history after the run using Rich libraryprint_history:True# disable python warnings if they annoy youignore_warnings:True# seed for random number generators in pytorch, numpy and python.randomseed:"0xCAFFEE"Experiment ConfigurationLocation:configs/experimentYou should store all your experiment configurations in this folder.Experiment configurations allow you to overwrite parameters from main project configuration.Simple example# @package _global_# to execute this experiment run:# python run.py experiment=example_simple.yamldefaults:-override /trainer:default.yaml-override /model:simple_unet.yaml-override /datamodule:mg_full.yaml-override /callbacks:default.yaml-override /logger:default.yaml# all parameters below will be merged with parameters from default configurations set above# this allows you to overwrite only specified parametersname:simple_exampleseed:"OxCAFFEE"trainer:epochs:5model:filters:16num_layers:4activation:"relu"datamodule:batch_size:8Project Organization├──.venv                <- Local poetry environment
    │   └──.gitkeep
    ├── configs
    ├── data
    │   ├── external        <- Data from third party sources.
    │   ├── interim         <- Intermediate data that has been transformed.
    │   ├── processed       <- The final, canonical data sets for modeling.
    │   └── raw             <- The original, immutable data dump.
    ├── logs                <- Tensorboard logging folder; Will be created by training
    ├── experiments         <- All your runs will be saved here; Will be created by training
    ├── mlflow              <- MLflow logging folder; Will be created by training
    ├── models              <- Trained and serialized models, model predictions, or model summaries
    ├── notebooks           <- Jupyter notebooks. Naming convention is a number (for ordering),
    │                         the creator's initials, and a short `-` delimited description, e.g.
    │                         `1.0-jqp-initial-data-exploration`.
    ├── references          <- Data dictionaries, manuals, and all other explanatory materials.
    ├── reports             <- Generated analysis as HTML, PDF, LaTeX, etc.
    │   └── figures         <- Generated graphics and figures to be used in reporting
    ├── test                <- Data dictionaries, manuals, and all other explanatory materials.
    ├── scalingtheunet                        <- Source code for use in this project.
    │   │
    │   ├── data                              <- Scripts to download or generate data
    │   │   ├── __init__.py
    │   │   └── make_dataset.py               <- TODO
    │   ├── dataloaders                       <- Scripts to handel and load the preprocessed data
    │   │   ├── __init__.py                   <- TODO
    │   │   └── hzg_mg_tomo.py                <- TODO
    │   ├── evaluation                        <- Scripts to do evaluation of the results
    │   │   └── __init__.py                   <- TODO
    │   ├── executor                          <- Scripts to train, eval and test models
    │   │   ├── __init__.py                   <- TODO
    │   │   ├── testing_9axes.py
    │   │   └── training.py
    │   ├── models                            <- Scripts to define model architecture
    │   │   ├── __init__.py                   <- TODO
    │   │   └── simple_unet.py
    │   ├── utils                             <- TODO
    │   │   ├── __init__.py                   <- TODO
    │   │   ├── file_system.py                <- TODO
    │   │   ├── logger.py                     <- TODO
    │   │   ├── losses.py                     <- TODO
    │   │   ├── metrics.py                    <- TODO
    │   │   ├── my_callback.py                <- TODO
    │   │   └── utils.py                      <- TODO
    │   │
    │   ├── visualization                     <- Scripts to create exploratory and results oriented
    │   │                                       visualizations
    │   └── __init__.py                       <- Makes {{ cookiecutter.module_name }} a Python module
    │
    ├── .editorconfig         <- file with format specification. You need to install
    │                             the required plugin for your IDE in order to enable it.
    ├── .gitignore         <- file that specifies what should we commit into
    │                             the repository and we should not.
    ├── LICENSE
    ├── Makefile            <- Makefile with commands like `make data` or `make train`
    ├── poetry.toml         <- poetry config file to install enviroment locally
    ├── poetry.lock         <- lock file for dependencies. It is used to install exactly
    │                         the same versions of dependencies on each build
    ├── pyproject.toml      <- The project's dependencies for reproducing the
    │                         analysis environment
    ├── README.md           <- The top-level README for developers using this project.
    └── setup.cfg           <- configuration file, that is used by all tools in this project