euske/pdfminer — pull request #15 by jcushman (2012-06-22 19:45:10+00:00)
Title: 2x faster layout analysis: Use set instead of list for Plane's internal collection of objects.

This patch changes utils.Plane._objs from a [] to a set(). In my tests, this results in layout analysis (PDFPageInterpreter.process_page) running more than twice as fast, and has no effect on the generated etree.

The reason this helps so much is that LTLayoutContainer.group_textboxes() runs Plane.__contains__() in a tight loop. For an ordinary PDF I tried, it ran over 5 million times per page. "obj in set()" turns out to be way faster than "obj in []"

Note: since the generated etrees are identical, I *assume* there's no reason for Plane._objs to be ordered, and I don't see any reason in the rest of Plane's code. But I'm not familiar with the whole code base, so check me on that.

Thanks!
Jack