tidy BOM handling and add a test
be more thorough about guessing delimiters