You are correct, it does allow for some invalid format specifiers to get through (so does gnuplot). The challenge is that without regex, if you truly want to model the syntax for format specifiers, you need to explicitly encode the languages's state machine.

One option would be to remove some of the formatting options, eg only allow 0-9.