Expand refs constraints to include valid characters

Fixes #4831, #4865, #4932