Add race conditions to config edit

Change-Id: I4030cd6466be86ed9890daae2daf3af1bdf8c27f
Fixes: rhbz#997941