Keep a reference count so we don't kill the JNI before we want it killed