Add spec case for complex regexp with binary encoding

Taken from issue #1669
Setup no encoding for escaped string properly

Fixes #1669