Are you creating an NMatrix of arrays here? That's what it looks like, and it seems unsafe.