jrochkind/traject — pull request #45 by billdueber (2013-10-09 15:42:18+00:00)
Title: Make compatible with non-JRuby implementations

So, I've been thinking about how there are parts of traject I'd like to use elsewhere (most notably the MarcExtractor and tmap stuff), and I've actually _been_ using traject for tangentially-related side projects -- looping through files to extract data for reports and such, and (on the other end) doing non-MARC indexing. And it struck me that I don't necessarily want to always use JRuby. [Well, _I_ usually do, but startup time is slow...]

So, there are a couple ways to go with this:
* suck it up
* split traject into a _core gem and a few other gems, since so little of it (thread pool, sorlj writer, marc4j reader) rely on JRuby. This is probably the smartest.
* run `if defined? JRUBY_VERSION` around the bits of traject that rely on JRuby

This pull request represents the third option. It took surprisingly little work (like, 20mn) to do so. All it really does is guard some of the includes, change the default reader/writer to Marc/Null, and force the number of processing threads to 0 if it's not jruby. Tests that rely on jruby have been renamed to `*_test_jruby.rb` and the rake task changed accordingly.

Sorry about the whitespace stuff; if we decide to go this direction I'll clean it up.

Having said that, I *don't* think this is the right way to go. I think it should be split up into gems.

If we go the gem route, I would think:

* traject_marc -- the marc reader and macros
* traject_jruby_threadpool -- the threadpool. Otherwise, no threading unless someone writes a non-java version
* traject_solrj -- the solrj writer and mock writer
* traject_marc4j -- the marc4j reader
* traject_core -- everything else
* traject -- just require all of the above

It's already easy to imagine things like `traject_db` for indexing from (or to?) a database, or `traject_elasticsearch` or `traject_rsolr` for alternate writer options. Someone could whip up a `traject_csv` reader and associated macros in a couple hours, and that includes lunch.

The hard part of splitting it up would be figuring out which tests go with which gems; the rest would be easy-peasy.

Thoughts on this? I imagine you're getting kind of burned out; you've been working on this crap longer than I have.