SciRuby/nmatrix — pull request #160 by cjfuller (2013-10-29 07:31:04+00:00)
Title: GC Management Changes (also list/yale inject rank, slicing bugfixes)

The GC management fixes are now up and running completely. The specific details of the final approach are in the individual commit notes and in the code documentation. To summarize, I've put in a static linked list that holds the addresses of all VALUEs in use, which is in turn wrapped in a ruby object so the GC knows to mark this linked list. The list in turn will mark all the VALUEs in use. This has the net effect of making sure that anything in use by NMatrix C functions is wrapped in a GC-visible/markable ruby object without the overhead of having to create a ruby object everywhere a VALUE is used in the C code.

Before *any* ruby C API call (or call to a function that calls the API, etc.), and there are a *lot* of them, the code must ensure that any VALUEs in use are registered in this linked list. Before that code returns, it must also unregister the VALUEs so that the GC can collect them at the appropriate time, or else memory is leaked (analogously to malloc() and free()). There's a lot of helper functions that have been added to ease this with various NMatrix-specific datatypes that might be holding values (e.g. `nm_register_value`, `nm_unregister_value`, `nm_yale_storage_register`, `nm_yale_storage_unregister`, etc.).

At some point in the next few days, I'll write up a more detailed explanation (and a howto for using these methods in new code) in the wiki so that this is thoroughly documented for the future. (Perhaps as the start of a new NMatrix developers' guide?)

I can't get GC-related segfaults any more in my complex testing code that uses dense matrices, so I believe this fixes #139. I don't have any particularly complicated code for testing sparse matrices, though, so while it hasn't broken anything there and I haven't seen any segfaults there either with the new fixes, it's possible I've missed something with the yale/list changes.

On my elementwise-op-heavy test code, there's a slowdown I'd estimate on the scale of 5% due to this additional GC tracking. Not sure if other code will be worse though... we'll have to keep an eye out.

This also fixes #152 and fixes #153. These are included in the same pull request as I found it difficult to get the specs for those running without fixing the segfaults, and I couldn't check I wasn't breaking things as I was fixing the segfaults without getting the specs for those issues working.