Forbid nested brackets in references

This applies to footnotes/abbrs as well, because they are
designed to be implemented in the similar way.