Issue: Peewee doesn't resolve DeferredForeignKey if class being referred to is already instanciated
Author: kkinder
State: closed
Created: 2019-03-20 19:12:05

Here's a quick example:

```
# models.py
import peewee

database = peewee.PostgresqlDatabase("foobar")
database.execute_sql('CREATE EXTENSION IF NOT EXISTS "uuid-ossp"')


class Artist(peewee.Model):
    id = peewee.UUIDField(primary_key=True)
    name = peewee.CharField()

    class Meta:
        database = database


class Album(peewee.Model):
    id = peewee.UUIDField(primary_key=True)
    artist = peewee.DeferredForeignKey('Artist')
    title = peewee.CharField()

    class Meta:
        database = database


print(peewee.DeferredForeignKey._unresolved)   # album.artist is unresolved

database.create_tables([Album, Artist])
```

In this instance, because Peewee *must* resolve the foreign key to figure out the type of the column, you'll get an exception:

```
peewee.ProgrammingError: syntax error at or near "NOT"
LINE 1: ...album" ("id" UUID NOT NULL PRIMARY KEY, "artist"  NOT NULL, ...
```

The reason is that this column was never resolved: `artist = peewee.DeferredForeignKey('Artist')`. It's still in the `DeferredForeignKey._unresolved` set, thus resulting in a syntax error upon table creation.

You might ask, if the table is already created as a class, why not use ForeignKey directly? The reason is that when you're splitting up a dozen or more tables, where each model gets its own file, it becomes too hard to figure out what's created first and what isn't. It's far easier to always use DeferredForeignKey and put dependent imports at the bottom.

I think the solution would be to have DeferredForeignKey check to see whether the class being referred already exists and if it does, immediately resolve it.

> The reason is that when you're splitting up a dozen or more tables, where each model gets its own file, it becomes too hard to figure out what's created first and what isn't. It's far easier to always use DeferredForeignKey and put dependent imports at the bottom.

What you're saying, in effect, is "I have a dependency graph but I'm too lazy to think about it".

You have foreign-key constraints in the database, foreign-key dependencies between model classes, and import-order dependencies between Python modules. If you get the import-order figured out, you should have no trouble getting the foreign-keys to work. If this seems too hard, I'd suggest you have too many Python modules, and you might consolidate them.

That being said, this feels kinda like a bug. The reason why Peewee doesn't resolve these is because Peewee doesn't keep an internal registry of every model subclass. Deferred foreign keys are resolved as a side-effect during model class creation, which typically happens at import time.

To fix this, I'd think that Peewee (or the DeferredForeignKey class) would need to track every model class that is defined, which doesn't seem great.

> What you're saying, in effect, is "I have a dependency graph but I'm too lazy to think about it".

Well, we're talking about the *order of imports* mattering. That is to say,  one of these might work, while the other will not:

```
from artist import Artist
from album import Album
```

vs:

```
from album import Album
from artist import Artist
```

That can result in a situation where all your unittests might clear, and then when you sort your imports alphabetically, it breaks, for example. Just broadly speaking, it's my feeling that the order you import modules shouldn't matter.

> To fix this, I'd think that Peewee (or the DeferredForeignKey class) would need to track every model class that is defined, which doesn't seem great.

Yeah, that's more or less the tree I was barking up too, and it doesn't seem great.

For the moment, my actual solution is, in the main file, is this:

```
for table in all_tables:
    DeferredForeignKey.resolve(table)
```

That verifies all the things are resolved.

*As an aside:* Arguably it's a separate bug that failing to resolve a DeferredForeignKey column results in a syntax error from the database because the column type definition is missing. But to get this to happen, I had to be using uuid's.


> Well, we're talking about the order of imports mattering. That is to say, one of these might work, while the other will not

Only because the resolution of the dependency graph has been pushed til later. Ordinarily, using regular `ForeignKeyField`, you would have to resolve your dependencies between the modules themselves in order to declare the `ForeignKeyField()` in the first place.

`DeferredForeignKey` really exists for one reason: to make it possible to declare circular foreign-key dependencies. This is described where it is documented,

* http://docs.peewee-orm.com/en/latest/peewee/api.html#DeferredForeignKey
* http://docs.peewee-orm.com/en/latest/peewee/models.html#circular-foreign-key-dependencies

I'd suggest ironing out your module organization.

Consider if this were not "peewee" but *literally any other python library* and you have all these interdependencies between the classes. You would need to resolve these types of issues if you wanted to break them out into a ton of little modules.

I think, however, you're over-simplifying the matter as inter-dependency between classes. It is, but only at create time.

If you have a fairly modular library, or a handful of libraries, managing dependencies is one thing, but managing import order is another, especially when you can't control in what order the consumers of your library might choose to import them. I'd argue that, just like how you maintain a list (on a class attribute) in one direction for deferred foreign keys, it would make sense to do it in the other way.

Having said that, if the official Peewee solution is just to be careful about in what order you import modules, and since most editors and programmers just go with alphabetical order (I think that's even in PEP8), I'll stick with manually resolving them inside the library to avoid the ambiguity. Thanks for the thorough reply!

Peewee model classes are apples-to-apples the same as any other python classes, functions, etc. If you have objects that depend on other objects within your collection of modules, you'll have to resolve those dependencies. Peewee models are the same as any other python objects in this regard.

The problem `DeferredForeignKey` was designed to solve is mutual dependence.

Using `DeferredForeignKey` to paste over a fragmented or poorly-thought-out dependency graph is possible, but never advisable.

For what its worth, when I have two modules that are somehow interdependent, the solution is usually to either move them into a single module - or create a third module that imports and resolves both deps.

Discussion with real examples available here: http://charlesleifer.com/blog/structuring-flask-apps-a-how-to-for-those-coming-from-django/

Fair enough. I think your update to the docs would have also kept me going down the rabbit hole I originally went down, which was one of getting sql syntax errors because the `DeferredForeignKey` was never resolved.

What I might say is that perhaps, *as a feature* some kind of `LazyForeignKey` that actually does what I'm describing would be useful. At least in my opinion.