setup.c: check that the pathspec magic ends with ")"

The previous code did not diagnose an incorrectly spelled ":(top"
as an error.

Signed-off-by: Andrew Wong <andrew.kw.w@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>