akumoth/peewee-issues huggingface dataset:
Issue: Enable use of range for IN filters in python 3
Author: conqp
State: closed
Created: 2020-10-07 12:47:41

While it is currently possible to do something like

`Model.select().where(Model.id << range(42, 256))`

in Python 2, this is not possible in Python3.
The reason being, that `range()` returns a `list` in Python 2, which is currently a member of `Value._multi_types`, but is of type `range` in Python 3.
Therefor I used the existing Python 2/3 branch logic to declare `multi_types` globally with other version-specific constants and used it in `Value` instead.

I did not yet add a test case for this, since `test_select_in_list_of_values` in `sql.py` currently only handles strings.

I've merged an equivalent patch. I had hoped to also support `map` but that's more like a generator and doesn't support multiple iterations, so that doesn't work. At any rate, thanks for the suggestion.
Issue: GeneratedHashField
Author: milesgranger
State: closed
Created: 2020-10-07 10:38:54

Hi! :wave: 

First, thank you so much for such a great library. :+1: 

Feature request :ribbon: 
---
I would like to make use of PostgreSQL's `GENERATED ALWAYS AS` statement, for computing a hashed key based on one or more other fields. Specifically for a generating a hash

Solution :question: 
---
I've provided a prototype which accomplishes this, but it could use some updates for generalization (maybe not hash, different types, etc). Do you think this is worth pursuing or maybe best left as a custom implementation for private use? I suppose it is similar to SqlAlchemy's [Computed](https://docs.sqlalchemy.org/en/13/core/defaults.html?highlight=server#computed-generated-always-as-columns)

F-Strings are not supported in Python 2.x and Python 3 < 3.6.
This is why the check fails.

I see that, thanks! :+1:  Didn't think it was worth fixing right away since I imagine changes will be needed anyway, if this is at all even wanted. 

I'm not interested in maintaining or supporting this, but feel free to offer it as a 3rd-party extension.

Thank you for your input.

I don't see anything similar to [Computed](https://docs.sqlalchemy.org/en/13/core/defaults.html?highlight=server#computed-generated-always-as-columns) offered by Peewee (at least I couldn't locate anything in the docs), or is this the right approach?

No but since Postgres, MySQL and Sqlite have roughly the same syntax it's probably worth adding. I will look into this.

As I was implementing this, I ran into a number of subtle differences between the Sqlite, MySQL, MariaDB and Postgres implementations. For now, I think users will be best-off using the existing `constraints=` parameter to specify a generated column definition, although the code is in `feature/computed` branch if you want to fork it.

```python

constraint = SQL("generated always as (first || ' ' || last)")
full_name = CharField(constraints=[constraint])
```

Lovely! Thanks for the tip, I really appreciate it.
Issue: delete().execute() on model object deletes all records from the table.
Author: p1v2
State: closed
Created: 2020-10-07 10:25:06

Hey. Recently I ran into issue, when accidentally dropping all records from the table.

if we have:
```
from peewee import Model

class Entity(Model):
    # fields list
    pass

# entity is single record from DB
entity = Entity.select()[0]

# entity delete is query that deletes all records in table
# next code line returns:
# ('DELETE FROM "entity"', [])
entity.delete().sql()

# And this code will actually do delete
entity.delete().execute()
# woop, no more any entity in DB
```

Any chance to change it, that delete method will delete single record.
This feels much more obvious.
Also Django-ORM does delete single record, when you call delete on single model object.

To delete a single model instance, you should use `.delete_instance()`.

The documentation covers this in detail: http://docs.peewee-orm.com/en/latest/peewee/querying.html#deleting-records
Issue: KeyError when trying to dynamically add pre-existing field with a new default
Author: bdoms
State: closed
Created: 2020-10-06 17:16:31

This is a funny situation, but essentially I'm trying to dynamically rebuild a database. It starts with a call to `generate_models` (which is totally amazing, thank you!) but that does not include any defaults. In order to add them back in, I'm looping through the fields and adding the default on the fly. Rather than manually messing with the `_meta` properties like `defaults`, `_default_callables`, etc. I decided to use `Metadata.add_field`, like so:

```python
for name, field in list(model_class._meta.fields.items()):
    field.default = datetime.utcnow
    model_class._meta.add_field(name, field)
```

I noticed that `add_field` starts by removing the old one if it exists, so I figured this was safe to use. But it generates a key error because the old field didn't have a default on it:

```python
  File ".../python3.8/site-packages/peewee.py", line 5912, in add_field
    self.remove_field(field_name)
  File ".../python3.8/site-packages/peewee.py", line 5961, in remove_field
    del self.defaults[original]
KeyError: <Field: model_name.field_name>
```

It seems like it'd be an easy fix to just add a single line to `remove_field` right before that, so it becomes:

```python
if original in self.defaults:
    del self.defaults[original]
```

Or even turn it into a `pop` like the line below.

If for some reason you don't think this is something you want to support then I did find a pretty easy and logical workaround:

```python
for name, field in list(model_class._meta.fields.items()):
    model_class._meta.remove_field(name)
    field.default = datetime.utcnow
    model_class._meta.add_field(name, field)
```

The default is only removed if the original field instance has a default, so this seems like something that only would come about in a very weird way. Going to pass on any changes.
Issue: on_conflict support in bulk_create
Author: AntonioL
State: closed
Created: 2020-09-25 11:59:27

Please provide mechanism to ignore conflicts when calling bulk_create.

My target database is PostgreSQL.

At th moment I am handrolling my own query.

I'm going to pass on implementing this as part of `bulk_create()` since this can be easily handled by:

```python
Model.insert_many([....]).on_conflict_ignore()
```

The problem arises w/postgres where the insert cursor returns IDs only for the rows that are created, which can't be mapped back to the actual objects that were inserted.
Issue: Documentation - Example of `choices` parameter.
Author: ghost
State: closed
Created: 2020-09-24 19:42:48

Hey there. Thank you for the simple ORM! It's ideal for a pypi package I am building that has no django/flask gui.

Do have an example of `choices=[(),()]` that could be added to the docs? It is not clear to me.


```
choices (list) – An iterable of 2-tuples mapping column values to display labels.
http://docs.peewee-orm.com/en/latest/peewee/api.html#fields
```

I swear I saw an example of this somewhere, but looking back through the ReadTheDocs and the `/examples` folder on GitHub I cannot find it. 




That's not the correct way. An iterable of 2-tuples looks like this:

```python
[('key1', 'Label 1'), ('key2', 'Label 2')]
```
Issue: peewee saves value of foreign key field wrongly
Author: hamidb80
State: closed
Created: 2020-09-22 13:50:32

Hello.

I think peewee saves the value of foreign key wrongly. Because:
user.id is an integer but in the token records, `user_id` saves as "t<user_id>".

```python
class User(BaseModel):
    name = CharField()
    identifier = CharField(index=True, unique=True)
    email = CharField(unique=True)

    password = CharField()

    registered_at = DateTimeField()

    """ 
    back ref fields:
    -- tokens
    """


class Token(BaseModel):
    user = ForeignKeyField(User, backref='tokens')
    code = CharField()

    created_at = DateTimeField()
    will_expire_at = DateTimeField()

```

data saves like:
```
USER TABLE
id | name | ...
1  |  hamid

RECORD TABLE
id | user_id | ...
1  |  t1 	  | ...
```

Also I have problem with this:
```python
token = Token.get_or_none(code=token_key)
print(token.user)
```

output of following code:
`
UserDoesNotExist('<Model: User> instance matching query does not exist:\nSQL: SELECT "t1"."id", "t1"."name", "t1"."identifier", "t1"."email", "t1"."password", "t1"."registered_at" FROM "user" AS "t1" WHERE ("t1"."id" = ?) LIMIT ? OFFSET ?\nParams: [\'t1\', 1, 0]')
`

But if I modify the `user_id` in the token table manually, everything works great.


python 3.8.2
database: sqlite
peewee: last stable version at the time of opening the issue
ubunru mate 20

I think you've got some issue with your code, your base model class, or something. What you're suggesting is that peewee is somehow fundamentally broken, and the tests are all passing:

https://travis-ci.org/github/coleifer/peewee
Issue: Request: Option to disable numbered table name aliases (t1, t2, t3... etc)
Author: ibushong
State: closed
Created: 2020-09-18 19:46:15

Is it possible to add an option to disable the automatic mapping of table names to numbered aliases? When debugging/reading raw queries, it's much easier to read something like ``WHERE (`mytable`.`name` == 'foo')`` instead of ``WHERE (`t3`.`name` == 'foo')``

quick hack:
```python
class AliasManager(object):
    ...
    def add(self, source):
        if source not in self.mapping:
            self._counter += 1
            # ORIGINAL self[source] = 't%d' % self._counter
            self[source] = source.__name__
        return self.mapping[source]
```

The only side effect of this (in my application at least, which is by no means exhaustive) is that I have to add an explicit alias for any uses of `Table.alias()`, e.g. `Table.alias('mytable_alias'`) (which I think is probably a good practice anyway).


I don't have any plans to add such, but you are welcome to write a monkey-patch for AliasManager.add and maintain it separately for your own debugging purposes.
Issue: Next result cached in Select.iterator()
Author: jacobian91
State: closed
Created: 2020-09-16 05:20:54

I ran into an unexpected behavior that didn't seem to match what _I assume_ the documentation is saying. 

In the documentation under [Iterating over large result-sets](http://docs.peewee-orm.com/en/latest/peewee/querying.html#iterating-over-large-result-sets) it states:

> To reduce the amount of memory used by peewee when iterating over a query, use the `iterator()` method. This method allows you to iterate without caching each model returned, using much less memory when iterating over large result sets.

The way I read that, which may be wrong, is that any changes made to the dataset will be taken into account when the `next()` call is made. However, the result I'm getting is that value produced by `next()` is _cached_ from the previous call, not _retrieved_ during the current call. Here is some minimal code to reproduce and `assert` the values.


```python
import peewee

database = peewee.SqliteDatabase(":memory:")

class BaseModel(peewee.Model):
    class Meta:
        database = database
class A(BaseModel):
    """test"""

database.create_tables((A,))

# Create three records.
for _ in range(3):
    A.create()
assert len(A) == 3 # Sanity check.

# Create the generator object.
iter_query = A.select().iterator()

# Deleting more than 1-next() away doesn't return the record.
A.delete_by_id(3)

# Deleting record right before next still returns the deleted record.
A.delete_by_id(1)
assert next(iter_query).id == 1

# Next record returned even though the table is empty.
A.delete_by_id(2)
assert len(A) == 0
assert next(iter_query).id == 2

# Deleted more than 1-next() away, the record isn't retrieved.
try:
    next(iter_query)
    assert 0
except StopIteration:
    print("Stop Reached")

```

If you read the peewee code for `CursorWrapper` you will see that peewee is not eagerly fetching anything. The behavior you are seeing is a side-effect of the interaction between pysqlite (the stdlib sqlite driver) and how sqlite operates. You can see that [here](https://github.com/coleifer/pysqlite3/blob/master/src/cursor.c#L710-L772) pysqlite fetches the next row.

I'd suggest that you're doing something weird and should probably a) read all rows first, or b) not modify them while iterating over the cursor. Ordinarily this isn't a problem if you are using separate connections to do the reading/modifying because this is either completely prevented or, if using wal-mode, the read operates on a snapshot. It only can occur if you're modifying the table and iterating at the same time, which seems problematic for what I hope are obvious reasons.

Thank you for the quick response! I figured it was something wrong with my understanding of this.

Thankfully I was not building actual code, just experimenting with the details of how the .iterator() worked. I was essentially wanting to see what would happen using different processes, and I didn't know about the other ways this is prevented. Thank you, and great work on this package!
Issue: Saving `only` no fields saves all fields.
Author: arel
State: closed
Created: 2020-09-16 05:19:45

## The issue

I use the pattern of saving only fields that have changed in a model, as such:

```python
obj.save(only=obj.dirty_fields)
```

However, if the `dirty_fields` value is an empty list, I would expect `save()` to save _no_ fields (i.e., be a no-op.) Instead, it saves _all_ the fields, which is counter-intuitive and can lead to problems if I don't explicitly check for an empty list.

I believe the issue is in the line below:

https://github.com/coleifer/peewee/blob/ce9375cb4b3fa64560e400932d86e2632c4a0586/peewee.py#L6521

Instead of checking the falseness of `only`, it might be more appropriate to check if it is None:

```python
 if only is None:
    ...
```

Thanks, I've made the change as suggested (although it is `if only is not None`). Peewee is active-record so calling save with no data will raise a `ValueError`.
Issue: Please add support for enums
Author: emdete
State: closed
Created: 2020-09-14 17:29:55

Postgresql allows to create enum-types like

```
CREATE TYPE colors AS ENUM ('red', 'green', 'blue');
```

which is quite useful. Its a feature i really would like to see supported in peewee.

Thank you! :) 

I have no plans at present to support this. It should be easy, if you so desire, to implement an `EnumField` as a separate class.
Issue: Support Meta inheritance
Author: jonashaag
State: closed
Created: 2020-09-11 16:34:10

Would be nice if this worked:

```py
class DefaultMeta:
    database = db
    ...

class Foo(Model):
    ...

    class Meta(DefaultMeta):
        ...
```

Right now it silently ignores the `DefaultMeta` fields.

The meta class is a weird kind of a hack inspired by similar deal in Django. The meta class is not what gets inherited, but instead the "inheritable" settings of the Meta class are inherited by the model.

Ex:

```python

class BaseModel(Model):
    class Meta:
        database = db

class Foo(BaseModel):
    class Meta:
        table_name = 'foo_tbl'
        # "database" setting is inherited from BaseModel.
```

This is described in the documentation for model options and table metadata:

http://docs.peewee-orm.com/en/latest/peewee/models.html#model-options-and-table-metadata

OK, that works for me, but it's different from Django. https://docs.djangoproject.com/en/3.1/topics/db/models/#meta-inheritance
Issue: Add support for select clause in bulk_update
Author: jonashaag
State: closed
Created: 2020-09-11 15:59:56

Add support for optional select clause in `bulk_update`, for example to allow for something like

```py
foos = [... list of Foos ...]
# Only allow updates to x when x=0.
nchanged = Foo.bulk_update(foos, [Foo.x], q=(Foo.x == 0))
if nchanged != len(foos):
    raise Error("Some Foos had x != 0")
```

I quickly hacked this together like this, and it seems to work.

```py
    def bulk_update(cls, model_list, fields, batch_size=None, q=None):
...

            query = (
                cls.update(update)
                   .where(cls._meta.primary_key.in_(id_list))
            )
            if q is not None:
                query = query.where(q)
            n += query.execute()
        return n
```

Since you are already explicitly specifying the `model_list` this seems redundant.

It's not – think of my use case as "defensive" bulk updating of an attribute, for example allow only changes to `x` if `x` wasn't ever set and still has its default value of 0.

I *think* this can be done in Django as follows, but I'm not entirely sure (haven't tried):

```py
Foo.objects.filter(x=0).bulk_update(foos, ['x'])
```

As an alternative, what do you think about factoring out the `Case` builder logic from `bulk_update` into a public helper method so it's easier for people to write their own custom bulk update logic?

I'm not clear why you would be using bulk_update in that case? Presumably a regular-old update() query would work better?

Because the `x` may be different for each `Foo`. Say we start with 3 `Foo` in the database, with `x`=0. We do some work that computes 3 new (different!) `x` values, and we want to update all of the `Foo`s with the new `x` values, with a single SQL update statement and in a way that never updates any `x` that were updated by something else while we were computing the new `x` values (race conditions).

A relational database provides atomicity and isolation, so presumably there's a different way to handle that. I think this is a rather obscure usage because it implies that you are updating values from a list of models, but also that some of those models may be "out-of-sync". This, to me, is not an API failure but rather a result of trying to shoehorn some logic into a process that would be better handled another way.

Isn't the scenario simply a case of a more general version of optimistic locking as described in the docs? http://docs.peewee-orm.com/en/latest/peewee/hacks.html#optimistic-locking There we have a version column, and no bulk update; here we have a non-version column with bulk update.
Issue: Limit on backrefs ?
Author: hrishikeshrt
State: closed
Created: 2020-09-10 20:39:47

When using `model_to_dict on` a model with possibly a large number of `backrefs`, could there be an argument to `model_to_dict` which limits the number of entries in any back reference to the given value? (or any alternate way to achieve the same.

Just define which models or foreign-keys to exclude and they won't be traversed. Alternatively, you can explicitly implement your serialization strategy.

Oh wait, I think I understand now -- no, this is something you will probably want to implement in a custom serializer if you want to control how the back-references are also queried.

Alright, thank you.
I am currently doing it as follows,

`occurrences` is my `backref`

```
    _result = MyModel.select().where(conditions).limit(limit)
    results = []
    for _result in _results:
        result = model_to_dict(_result)
        result['occurences'] = [
            model_to_dict(occurence, backrefs=True)
            for occurence in _result.occurences.limit(limit)
        ]
        results.append(result)
```

I suppose this is the proper / (only?) way then?

That's perfectly fine way, and what I would probably do as well. The next step up would be a dedicated serialization library.
Issue: 'Playhouse" Feature request: Add support for Postgresql Range types
Author: bjmc
State: closed
Created: 2020-09-09 18:33:12

Postgresql range types [are fantastic](https://wiki.postgresql.org/images/7/73/Range-types-pgopen-2012.pdf) and it would be great to be able to use them directly in Peewee.

No plans to support this, but feel free to implement as a 3rd-party extension.
Issue: Question: get chagnes
Author: mrquokka
State: closed
Created: 2020-09-02 11:49:26

I look for orm with work with save steps to apply it later.
I need to get all changes what happend on python, but dont sended to database.
Can i get simular of this?

I'm not sure how to answer this, but I'd suggest providing examples and asking on stackoverflow. This sounds more like a question rather than a peewee-specific issue.
Issue: Is it possible to modify the columns for a TableFunction in SQLite at runtime?
Author: pacuna
State: closed
Created: 2020-08-31 20:47:20

I'm trying to create a table function that generates columns depending on the input. For example if the input is `("a", 1, "b", 2)` it should generate columns **a** and **b** with values **1** and **2**.  My code is working but I have to hardcode the columns in the class variable `columns` and I can't modify it after. Is there a way to accomplish this?

Thanks. 

Not really, look at the docs for the sqlite virtual table interface if you're curious how it works:

https://sqlite.org/vtab.html

The virtual table, when registered, has to declare it's structure.
Issue: Adding a new ForeignKeyField to an existing model gives an error
Author: fmors
State: closed
Created: 2020-08-28 14:19:55

I've noticed that, when you add a new field to an already existing model, it will try to do all steps to create a table:

1. create sequences
2. create table (proper)
3. create indexes

However, the "safe" parameter will make sure that the schema of the table itself will not be changed (after all, the table exists). As a result, the new field will not be created. The migrator does a good job creating the fields that you need.

But, if the field is a ForeignKeyField, it will try to create an index for that field (step 3 above). Because the field was not created on step 2, we will have an SQL error about a non-existing field.

Maybe there should be a check if the field exists before trying to create the index. Because it is a new field that will be dealt by the migration steps (using the SchemaMigrator), the index creation should also be done at around this moment.

Peewee does not have any automatic schema migration when you change a model definition. You can write migrations quite easily using the playhouse extension:

http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#migrate

Do I understand correctly that you're describing:

1. changed your model definition
2. ran `create_tables()`
3. table already existed, but then the index is attempted to be added
4. error occurs because the column does not exist

If that's the case then I'd propose this isn't a bug, but rather that you should migrate your schema up beforehand.

That's right, indeed by migrating the schema before anything (including creating any new tables beforehand), the create_tables() thing will when work correctly. Thanks!
Issue: Custom SQL
Author: ghost
State: closed
Created: 2020-08-24 03:30:33

Hello...

Who to change?

    SELECT "t1"."id", "t1"."text" FROM "fields" AS "t1" WHERE ("t1"."id" = 1) LIMIT 1

For:

    SELECT FIRST 1 "t1"."id", "t1"."text" FROM "fields" AS "t1" WHERE ("t1"."id" = 1)

Thanks


That syntax is not currently supported out of the box. I'd suggest looking at the query builder internals.

If you have more questions feel free to ask on the mailing list or #peewee on freenode and I'll try to help.

ok, thanks.