I follow an advise from here: http://stackoverflow.com/questions/5821238/rake-dbcreate-encoding-error-with-postgresql

Basically they suggest to recreate the template1, but given the complexity of that, there is a comment that suggests using template1. And that works for me.