RDFLib/rdflib — Issue #146: Turtle Files with a UTF-8 BOM fail to parse
Author: gjhiggins
Created: 2012-02-20 08:33:47+00:00
State: open

rve...@gmail.com, 2011-01-04T13:31:16.000Z

What steps will reproduce the problem?
1. Try to parse a Turtle file which has a UTF-8 BOM at the start

What is the expected output? What do you see instead?

File should parse without issue but instead parsing fails - am reporting this issue on behalf of one of my users so haven't reproduced this myself

UTF-8 BOMs are not required but they are permitted so the parser is rejecting perfectly valid files.

This issue will likely affect all UTF-8 formats (Notation 3, TriG, SPARQL Query etc)

Comment 1 by azarot...@gmail.com
Confirmed this bug, and have a fix:

in plugins/parsers/notation3.py, line 910 and following:

if not isinstance(octets, unicode):
str = octets.decode('utf-8')
# NB Already decoded,so \ufeff
if str[0] == codecs.BOM_UTF8.decode('utf-8'):
str = str[1:]
else:
str=octets

And import codecs at the top.

After this change, the test file submitted by rvesse above works:

f = file('ttl-with-bom.ttl')
bom = StringIO(f.read())
f.close()
bomg= ConjunctiveGraph()
bomg.parse(bom,	format='n3')

Comment 2 by ed.summers

Comment 3 by ed.summers
This issue was closed by revision r1900.

Comment 4 by ed.summers
This issue was closed by revision a83de1008e93.