Handle regexp replacements with $1, $2, etc. properly

For #5275. Supersedes #5295.