Yes, if we want to allow empty values and take them as "use default", then we do need to test if an option is set *and* if it is non-empty.
I think that is still better than filling the config with default values.

Additionally, I am not sure if we should allow empty options and take them as defaults.
Failing with an error if an option is empty is also a possibility. Either way, we should test if the option is empty.

If you think it is a bit redundant to mention every option in two tests and when changing a default (3 times in 2 lines), then use a helper function that does both checks, given config, section and option as input. This is probably the way to go if we want to print an error message.

Right now I tend to not allow setting empty options (and printing an error if it is empty).
I don't really see the use of setting options to empty (rather than just to comment them out or removing them from a config file).