How to save a copy of the running script into the log folder?
Hi,
I want to save a copy of the running script of each experiment.
I have tried to apply the following callback:
import os
from pathlib import Path

class MyCopyingCallback(pl.Callback):

    def on_init_end(self, trainer):
        log_dir = trainer.logger.log_dir
        Path(log_dir).mkdir(parents=True, exist_ok=True)
        shutil.copy2(os.path.realpath(__file__), os.path.join(log_dir, os.path.basename(os.path.realpath(__file__))))

The problem is that I ended up with two copies of my running script under two different folders (I'm using the ddp backend)
Any ideas on how to solve this issue?
Hey!
You can use the rank id to determine in which process you are, like so:
if trainer.global_rank == 0:
    # do something only on first process
In your example, this could make sense:
import os
from pathlib import Path

class MyCopyingCallback(pl.Callback):

    def on_init_end(self, trainer):
        if trainer.global_rank != 0:
            return 
        log_dir = trainer.logger.log_dir
        Path(log_dir).mkdir(parents=True, exist_ok=True)
        shutil.copy2(os.path.realpath(__file__), os.path.join(log_dir, os.path.basename(os.path.realpath(__file__))))