Show properties config button if the form exists

Fixes #934