Issue: Migrator missing not null column during column removal
Author: frdfsnlght
State: closed
Created: 2018-12-10 17:35:47

I have a model I've been using successfully for while:

```python
class Drink(Model):
    primaryName = CharField()
    secondaryName = CharField(null = True)
    glass = ForeignKeyField(Glass, backref = 'drinks')
    instructions = TextField(null = True)
    isFavorite = BooleanField(default = False)
    isAlcoholic = BooleanField(default = True)
    isOnMenu = BooleanField(default = False)
    timesDispensed = IntegerField(default = 0)
    source = CharField(default = 'local')
    createdDate = DateTimeField(default = datetime.datetime.now)
    updatedDate = DateTimeField(default = datetime.datetime.now)
```

I'm trying to remove and add some columns with the migrator:

```python
    drink_numIngredients = IntegerField(default = 0)
    drink_numIngredientsAvailable = IntegerField(null = True)
    
    migrate(
        migrator.drop_column('drink', 'createdDate'),
        migrator.drop_column('drink', 'updatedDate'),
        migrator.add_column('drink', 'numIngredients', drink_numIngredients),
        migrator.add_column('drink', 'numIngredientsAvailable', drink_numIngredientsAvailable),
    )
```

I have debug logging turned on and I can see this right before the exception:

```
2018-12-10 12:19:23,249 [MainThread] [peewee] [DEBUG]: ('CREATE TABLE "drink__tmp__" ("id" INTEGER NOT NULL PRIMARY KEY, "primaryName" VARCHAR(255) NOT NULL, "secondaryName" VARCHAR(255), "glass_id" INTEGER NOT NULL, "instructions" TEXT, "isFavorite" INTEGER NOT NULL, "isAlcoholic" INTEGER NOT NULL, "isOnMenu" INTEGER NOT NULL, "timesDispensed" INTEGER NOT NULL, "updatedDate" DATETIME NOT NULL, "source" VARCHAR(255) NOT NULL, FOREIGN KEY ("glass_id") REFERENCES "glass" ("id"))', [])
2018-12-10 12:19:23,251 [MainThread] [peewee] [DEBUG]: ('INSERT INTO "drink__tmp__" ("id", "secondaryName", "glass_id", "instructions", "isFavorite", "isAlcoholic", "isOnMenu", "timesDispensed", "updatedDate", "source") SELECT "id", "secondaryName", "glass_id", "instructions", "isFavorite", "isAlcoholic", "isOnMenu", "timesDispensed", "updatedDate", "source" FROM "drink"', [])
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/peewee.py", line 2653, in execute_sql
    cursor.execute(sql, params or ())
sqlite3.IntegrityError: NOT NULL constraint failed: drink__tmp__.primaryName
```

I can see the 'createdDate' column is removed during the temp table creation, and then peewee wants to fill the temp table with the data from the original, but it doesn't include the 'primaryName' column in the select resulting in the NOT NULL constraint failure.

Any ideas?


Oh, I should have mentioned, I'm using SQLite.

Ahh, yeah, its these lines that are causing the issue:

https://github.com/coleifer/peewee/blob/3634aea5ece71e21a2ac5cdb2b74b194e8257960/playhouse/migrate.py#L682-L685

It appears that if I rename my 'primaryName' column, the problem goes away (as long as the new name doesn't start with 'primary' (or 'foreign')).

I suppose I could do that, but of course it will be a PITA.

For now, I implemented raw ALTER TABLEs as necessary.  I'll have to think this over and figure out if it's worth renaming the column.


Fixed in 1e00d33