tenderlove/nokogiri — Issue #565: HTML Parsing Incorrectly with Nokogiri 1.5.0 and libxml2 2.6.26
Author: aselder
Created: 2011-11-10 06:28:13+00:00
State: open

I'm trying to use Nokogiri to parse the status page of my haproxy page, but the resulting Nokogiri document in truncated and mangled.

irb(main):003:0> Nokogiri::VERSION_INFO
=> {"warnings"=>[], "ruby"=>{"engine"=>"mri", "version"=>"1.8.7", "description"=>"ruby 1.8.7 (2009-06-12 patchlevel 174)
[i686-linux]", "platform"=>"i686-linux"}, "libxml"=>{"loaded"=>"2.6.26", "binding"=>"extension", "compiled"=>"2.6.26"},
"nokogiri"=>"1.5.0"}

doc = Nokogiri::HTML(open('http://www.blahblah.com:blahport/haproxy-status', :http_basic_authentication => ['blah', 'blah']))

=> https://gist.github.com/1354253

Notice that are only two elements to the children array, a DTD and an XML comment

When I switch to a different machine with a newer version of libxml2, the document parses correctly

Here's the nokogiri version info for a correctly running version.

{"warnings"=>[], "nokogiri"=>"1.5.0", "ruby"=>{"version"=>"1.9.2", "platform"=>"x86_64-darwin11.1.0", "description"=>"ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-darwin11.1.0]", "engine"=>"ruby"}, "libxml"=>{"binding"=>"extension", "compiled"=>"2.7.3", "loaded"=>"2.7.3"}}

I know the ruby version is different, but I noticed the same behavior in ruby 1.9.2-p290 with the older libxml.

I'm sorry, but I can't publicly post the HTML source or the correctly parsed Nokogiri document for security reason, but if one of the maintainer is looking at the bug, I can provide this data privately.