jrochkind/traject — pull request #10 by billdueber (2013-08-27 13:40:18+00:00)
Title: Replace spec_covering_field with something more efficient

I was going through marc_extractor just to understand what it's doing and noticed that the each_matching_line method wasn't taking advantage of ruby-marc's MARC::Record#fields(taglist). I ran a quick profile and saw that all those calls to #spec_covering_field were, in fact, taking up a whole bunch of time.

So I pulled out the #spec_covering_field logic and rearranged things a bit to short-circuit the conditionals. As near as I can tell, just this one little change nets about a 12% speed increase in MARC processing time (as measured by using the JSONWriter > /dev/null)

The bigger problem is that you're creating the spec every single freakin' time it's used. It would be better to have the spec generated once, and then just pass in the specific MARC::Record object. Should save a lot on garbage collection, because right now you're making and throwing away a lot of objects. But this is good for now.