(#21520) Provide option for soft failure during terminus writes when PuppetDB down

Signed-off-by: Ken Barber <ken@bob.sh>