Issue: Registering and accessing Sqlite modules after connection to db
Author: astrilchuk
State: closed
Created: 2018-07-04 17:58:48

This isn't a bug but a suggestion for possible improvement:

I recently ran across a scenario where I needed to conditionally register a Sqlite module for access to a virtual table.  But I realized that I couldn't register the module after the database connection had been established.  I also realized that even if I could register the module before connecting, I didn't have direct access to the module instance (I would have to keep my own reference to the module).

In summary, it would be nice to be able to create a module on-the-fly and access its instance via the connection object - since it's held anyway in a private dictionary.

Thanks

Are you referring to the "modules" functionality in `apsw`? Or the `TableFunction` APIs? Or loading a shared-library sqlite3 module?

I'm using the `apsw.createmodule()` functionality in order to register a virtual table so I can import bulk data from external sources.  Ideally I want to use a context to register the module, create the virtual table, import data, and finally drop the virtual table - though this functionality isn't what I'm asking for.  I just would like the ability to create the module after the db connection has been established and perhaps reference the instance of the module from the connection object.

I hope that makes sense.

If you look at the `register_module()` source code, it *should* be creating the module on the currently-open connection (and subsequent connections).

```python
    def register_module(self, mod_name, mod_inst):
        self._modules[mod_name] = mod_inst
        if not self.is_closed():
            self.connection().createmodule(mod_name, mod_inst)
```

If you want to only use the module for a single connection, you can instead:

```python
mymod = MyModule()
apsw_db = APSWDatabase(...)
apsw_db.connect()

apsw_db.connection().createmodule('mymod', mymod)
```

> In summary, it would be nice to be able to create a module on-the-fly and access its instance via the connection object - since it's held anyway in a private dictionary.

It seems weird to me that you'd want to access the module *instance* since theoretically the instance should be opaque and is only accessed through a virtual table.

When I attempt to add a module after a connection has been established an exception is raised: `"apsw.MisuseError: MisuseError: not an error"` originating from line 40 of `apsw_ext.py`.  It's very odd that it says "not an error" along with "MisuseError".  If I add the modules before connecting it works just fine.

The reason I access the module instance is because I dynamically change its data source.  i.e. I request multiple batches of external json data, assign the dictionary to the module, and perform a bulk insert.  Then rinse and repeat.

What happens if you just use plain old APSW?

I think you have a problem with your module implementation. I've tested locally using the example virtual table in the APSW docs and it works regardless of how/when I register it.

I'm starting to come to the same conclusion but I'm not sure what I'm doing wrong.  As long as I register the modules before connecting everything works 100%.  I'll continue to investigate on my side and update when I have more information.

My point is that this isn't a peewee issue.

I never suggested this was a peewee issue.  My original suggestion was that the modules be made publicly accessible.  This suggestion is regardless of whether or not I'm correctly registering the modules.  If you believe the modules should remain private, then I'm fine with this issue being closed.

For sure -- I think the traceback and behavior you're running into may be symptomatic of misusing the modules API however. If you wish to keep an instance handy that's of course possible, but my sense is that they are designed to be called by SQLite and should not exactly be "stateful" or mutable -- and exposing them via a public API might encourage that notion in other people.

It seems that I was somehow attempting to register the module twice which was causing the error.  Funny thing is that I can't reproduce it now.  I suspect I had some left-over debug code that has since been commented-out or deleted. 

And while investing the error it occurred to me that I just need to subclass the APSWDatabase to achieve what I want with accessing the instance.  Sometimes I'm so focused on a particular solution that I miss the most obvious one staring me in the face.

I'm sorry to have lead you on this goose chase. 

> I'm sorry to have lead you on this goose chase.

Not at all - it's a process of mutual discovery. The library got a new test as well, so that's a win.