Avoid repeated overwrites of config['extra_*']

Fixes #133
Well that makes it pretty clear. I'll update it. Thanks for the catch!