akumoth/peewee-issues huggingface dataset:
Issue: FOR UPDATE not supported by CockroachDatabase
Author: arel
State: closed
Created: 2023-03-18 02:06:31

I believe this is a bug. I am getting the following error when I try to use `for_update()` in Cockroach DB:

```
  File "/workspace/nuhom-api/.venv/lib/python3.10/site-packages/peewee.py", line 2509, in __sql__
    raise ValueError('FOR UPDATE specified but not supported '
graphql.error.graphql_error.GraphQLError: FOR UPDATE specified but not supported by database.
```

It looks like `CockroachDatabase` believes `FOR UPDATE` is not supported (which may have been the case in an earlier version): https://github.com/coleifer/peewee/blob/b5cee5e99bcc0b20010646cb52c60f8e37dd620a/playhouse/cockroachdb.py#L65

But, in the CockroachDB docs, it states that `FOR UPDATE` is now supported: https://www.cockroachlabs.com/docs/stable/select-for-update.html

And, even though CockroachDB does not support  [locking strengths](https://www.cockroachlabs.com/docs/v22.2/select-for-update#locking-strengths) (`FOR SHARE` or `FOR KEY SHARE`), the syntax is also supported to be compatible with Postgres.


Thank you for letting me know, I'll get this fixed.

In the meantime you can set the flag manually when you do your database initialization.

```
db = ...
db.for_update = True
```

Thanks!

When I enabled the flag the crdb tests are locking up and failing, so something must be special about their implementation. If you're able to look into it that'd be great but if not I'll get around to it. (runtests.py -e cockroachdb)

I'm just marking the locked tests to skip for CockroachDB. I have no clue why it isn't working like Postgres or MySQL. This should be resolved now by:

0031ca565651bc3fd4a12ba347c4cb76c99a643f and 7357d9303a07826bab539a508f05b534f6fcbcc1
Issue: `CharField.max_length` has no effect on a Sqlite database
Author: calvinweb
State: closed
Created: 2023-03-17 15:23:13


That is correct, and is down to the way SQLite implements text affinity columns.

If you want to enforce a max length you can add a check constraint or implement a subclass of charfield that overrides the `db_value()` method to trim the string.

> That is correct, and is down to the way SQLite implements text affinity columns.
> 
> If you want to enforce a max length you can add a check constraint or implement a subclass of charfield that overrides the `db_value()` method to trim the string.

but i think this should be done by peewee itself because databases should have same behavour
Issue: Creating indexes on UUID primary key failes
Author: martijnED
State: closed
Created: 2023-03-14 14:19:57

During creating of tables and creating indexes on that table it failes to create indexes on UUID key.
## The code 
``` python 
import uuid
from playhouse.postgres_ext import  UUIDField

class CommandControl(Model):
    id = UUIDField(primary_key=True, default=uuid.uuid4)

def migrate(migrator, database, fake=False, **kwargs):
    """Write your migrations here."""
    migrator.create_table(CommandControl)
    migrator.add_index(CommandControl.id)

``` 
## The error
``` bash
  File "/app/__pypackages__/3.10/lib/peewee_migrate/migrator.py", line 117, in wrapper
    model._meta.schema = migrator.schema
AttributeError: 'UUIDField' object has no attribute '_meta'
```

The docs are pretty clear:

```python
# Adding an index:

# Specify the table, column names, and whether the index should be
# UNIQUE or not.
migrate(
    # Create an index on the `pub_date` column.
    migrator.add_index('story', ('pub_date',), False),

    # Create a multi-column index on the `pub_date` and `status` fields.
    migrator.add_index('story', ('pub_date', 'status'), False),

    # Create a unique index on the category and title fields.
    migrator.add_index('story', ('category_id', 'title'), True),
)
```

Please at least look at the docs before opening an issue in the future.

On closer inspection it looks like this isn't even for peewee, but for a third-party package. Please refer to the 3rd party package maintainer's documentation or take it up with them.

If I do so, I get a different error:

## Code
``` python
import uuid
from peewee import UUIDField,


class CommandControl(Model):
    id = UUIDField(primary_key=True, default=uuid.uuid4)

def migrate(migrator, database, fake=False, **kwargs):
    """Write your migrations here."""
    migrator.create_table(CommandControl)
    migrator.add_index('command_control', ('id',), False)

```

## The error than
``` bash
TypeError: not all arguments converted during string formatting
```
bit unclear what this means and how to solve that part

You aren't showing what `migrator` is but from your earlier comment I inferred it is some 3rd party library `peewee_migrate`? Please refer there.

You completely right! Will ask the question there!
Issue: reflection.generate_models BinaryJSONField column always has index, even when underlying database schema has no index
Author: rberman
State: closed
Created: 2023-03-10 17:16:36

I have a table defined as follows:
```python
from playhouse.postgres_ext import BinaryJSONField
class MyTable(flask_db.Model):
    some_data = BinaryJSONField(index=False)
```
For this table, we don’t need an index since we only ever write to this table. 

When I try to use generate_models on this table, the generated BinaryJSONField has `index=True`, as demonstrated below:
```python
from playhouse.reflection import generate_models
from models import MyTable
generated_model = generate_models(flask_db.database, table_names=["mytable"])["mytable"]
generated_column = generated_model._meta.columns["some_data"]
reference_column = MyTable._meta.columns["some_data"]
# Raises an error:
assert generated_column.index == reference_column.index
```
I believe this is happening because `BinaryJSONField` inherits from `IndexedFieldMixin` and, therefore, defaults the `index` parameter to `True`. The reflection code checks for indexes and adds `index=True` to column definitions when necessary to create the corresponding indexes, but does not add `index=False` when they are not required ([see here](https://github.com/coleifer/peewee/blob/3fc96dbe33500d3e5099ea11ef7af080900127ba/playhouse/reflection.py#L762-L767)). That normally works, but not for column types that include `IndexedFieldMixin`. 

The preferred behavior would be for the `index=False` value to taken into account when creating the table. Would you please take a look? Thank you!
Issue: Prefetch with many ForeignKeyField to same model
Author: dim49v
State: closed
Created: 2023-03-10 13:06:12

Hello!
I'm have 2 models:
```
class State(Entity):
    uuid = UUIDField(column_name='uuid', primary_key=True)

class Transition(Entity):
    uuid = UUIDField(column_name='uuid', primary_key=True)
    source_state = ForeignKeyField(State, column_name='source_state_id', backref='_db_source_transitions')
    target_state = ForeignKeyField(State, column_name='target_state_id', backref='_db_target_transitions')
```
I'm trying load `State` entity and all his `Transition`:
```
states = State.select().where(...)
transitions = Transition.select()
states = prefetch(states, transitions, prefetch_type=PREFETCH_TYPE.JOIN)
```
I get the following query:
```
SELECT ... FROM "transition" AS "t1" 
INNER JOIN (SELECT "t2"."uuid", "t2"."uuid" FROM "state" AS "t2" WHERE (...)) AS "t3" 
ON (("t3"."uuid" = "t1"."source_state_id") OR ("t3"."uuid" = "t1"."target_state_id"))
```
and error in ON statement:
```
column reference "uuid" is ambiguous
```
Is there a way to specify the single foreign key used or is it a bug?

Seems to be working OK for me:

```python
class State(db.Model):
    name = TextField()
    def __str__(self):
        return self.name

class Transition(db.Model):
    source_state = ForeignKeyField(State, backref='sources')
    target_state = ForeignKeyField(State, backref='targets')
    def __str__(self):
        return '%s->%s' % (self.source_state, self.target_state)

db.create_tables([State, Transition])

s1 = State.create(name='s1')
s2a = State.create(name='s2-a')
s2b = State.create(name='s2-b')
s3 = State.create(name='s3')
Transition.create(source_state=s1, target_state=s2a)
Transition.create(source_state=s1, target_state=s2b)
Transition.create(source_state=s2a, target_state=s3)
Transition.create(source_state=s2b, target_state=s3)

query = State.select().where(State.name != 's3')
transitions = Transition.select()
p = prefetch(query, transitions, prefetch_type=PREFETCH_TYPE.JOIN)
for row in p:
    print(row.name, row.sources, row.targets)
```

Output as expected:

```
s1 [<Transition: s1->s2-a>, <Transition: s1->s2-b>] []
s2-a [<Transition: s2-a->s3>] [<Transition: s1->s2-a>]
s2-b [<Transition: s2-b->s3>] [<Transition: s1->s2-b>]
```

The above code also gives an error:
```
psycopg2.errors.AmbiguousColumn: column reference "id" is ambiguous
LINE 1: ...AS "t2" WHERE ("t2"."name" != 's3')) AS "t3" ON (("t3"."id" ...
                                                             ^
```
Query:
```
('SELECT DISTINCT "t1"."id", "t1"."source_state_id", "t1"."target_state_id" FROM "transition" AS "t1"
 INNER JOIN (SELECT "t2"."id", "t2"."id" FROM "state" AS "t2" WHERE ("t2"."name" != %s)) AS "t3"
 ON (("t3"."id" = "t1"."source_state_id") OR ("t3"."id" = "t1"."target_state_id"))', ['s3'])
```
Сhecked on the latest version 3.16.0.  
Database - PostgreSQL 10.23

Oh no, I tested with SQLite so I assumed it was ok. Reopening and will look into it, thanks for letting me know.

This is now fixed, thanks again.
Issue: Is it possible to serialise peewee.Expression?
Author: vdytyniak-exos
State: closed
Created: 2023-03-09 11:19:40

it is not an issue, but I didn't find other communication channel to get some ideas in how to serialise peewee.Expression to pass it to the API and later use there as filter in the query. Thanks

No, not really or not officially. You can see how the [dynamic filters work in flask-peewee](http://docs.peewee-orm.com/projects/flask-peewee/en/latest/admin.html#filters): https://github.com/coleifer/flask-peewee/blob/master/flask_peewee/filters.py

Probably the easiest thing to do though is to use the django-style query apis, eg `field__eq=value`. These can be passed to `model.filter()` like Django.

Lastly, we have irc on libera #peewee, or you can ask general questions on stack overflow. I regularly check SO and try to provide help there just fyi.

@coleifer Thanks for you answer. I am trying to implement my own json decoder, looks like it is that hard to represent pewee.Expression tree in json.
Issue: `playhouse.signals.Signal.disconnect` disconnects more than key, sender pair.
Author: real-yfprojects
State: closed
Created: 2023-03-05 08:30:29

When connecting two receiver to the same sender and disconnecting one of them. Both are disconnected. This bug is also illustrated by the code itself:

https://github.com/coleifer/peewee/blob/f88fa44a29601ca2de9fe492fda11b1fb28fb412/playhouse/signals.py#L25-L38

The `_receivers` set stores pairs of receiver and model. When disconnecting a single pair is removed if present. However `_receiver_list` looses all entries for which `n != name and s != sender` is false. Quick transformation:

$$ \lnot( n \ne name \land s \ne sender ) \Leftrightarrow $$

$$ \lnot( n \ne name ) \lor \lnot( s \ne sender ) \Leftrightarrow $$

$$ n = name \lor  s = sender $$ 

Notice that all receivers are removed that have a matching name or receiver.

Oh yes it should be instead:

```
if not (n,s)==key
```

Nice! Thanks for fixing. Do you have a rough estimate for the next release date?

Let me think on it.
Issue: Undocumented parameter `sender` for `playhouse.signals.Signal.disconnect`.
Author: real-yfprojects
State: closed
Created: 2023-03-05 08:03:44

https://github.com/coleifer/peewee/blob/f88fa44a29601ca2de9fe492fda11b1fb28fb412/playhouse/signals.py#L25

There is a `sender` parameter. However there is no such parameter here:

https://github.com/coleifer/peewee/blob/f88fa44a29601ca2de9fe492fda11b1fb28fb412/docs/peewee/playhouse.rst?plain=1#L2740

You just don't want to update/improve the documentation of your library or why are you closing it without a comment. Others will find that pretty unwelcoming.

7246a7f2b8c5adbae33180af3c817601560dc33a
Issue: Cast Bigint to timestamp
Author: antoinebrtd
State: closed
Created: 2023-03-03 23:16:36

Hello,

I am trying to partition some data using `fn.date_part` on a `TimestampField`.
Here is my simplified model:
```
class Game(Model):
    id = PrimaryKeyField()
    ts_played = TimestampField(utc=True, default=None, null=True)
    score = IntegerField(null=True)
```

My full query:
```
results = Game \
   .select(
       fn.date_part("week", Game.ts_played).alias("week"), 
       fn.SUM(Game.score).alias("total_score")
   ) \
   .where(
       Game.ts_played.is_null(False), 
       Game.score.is_null(False),
       Game.ts_played >= 123456789,
       Game.ts_played <= 987654321
   ) \
   .group_by(fn.date_part("week", Game.ts_played)) \
   .order_by(fn.date_part("week", Game.ts_played))
```

I get the following error:
```
peewee.ProgrammingError: function date_part(unknown, bigint) does not exist
```
which does not surprise me because TimestampField is not really a timestamp but rather a Bigint.

I then try to cast the underlying bigint to timestamp, replacing `Game.ts_played` everywhere in the query above with:
```
Game.ts_played.cast("timestamp")
```

I get the error
```
peewee.ProgrammingError: cannot cast type bigint to timestamp without time zone
```


It works fine without casting when I change `ts_played` to be a `DateTimeField`. Is there a convenient way to partition a `TimestampField` in weeks?

Is there a very compelling reason why you are using `TimestampField` at all, instead of `DateTimeField`? It's really intended mostly for Sqlite users, which does not provide a dedicated date/time type.

At any rate, you need to use the postgres `to_timestamp` function here if you insist on doing it this way:

```sql
select extract(week from to_timestamp(1677887000));
```

So probably something like (untested):

```python
.group_by(fn.date_part('week', fn.to_timestamp(Game.ts_played)))
```
Issue: Can't query with unicode character
Author: groggyegg
State: closed
Created: 2023-02-26 00:49:33

I have a column containing this type of data `{"zh": "\u65e5\u97e9\u5267"}` and when I query it with this code `Video.select().where(Video.director.contains('\u65e5\u97e9\u5267'))` , I'm not getting any result. It only works when I query non-unicode text e.g. `Video.select().where(Video.director.contains('"zh"'))`.


This probably has to do with the collation you're using. Is this MySQL?

No, I’m using SQLite

Wait so your data is *literally* the Unicode escapes? If so have you tried double escaping the forward quote?

> Wait so your data is _literally_ the Unicode escapes? If so have you tried double escaping the forward quote?

That seems to fix the problem. I'm suprised that I didn't realized that :astonished:
Thanks!
Issue: How to use with pgbouncer?
Author: backpropogation
State: closed
Created: 2023-02-21 11:23:00

Could someone, please, make clear, how to use peewee with pgbouncer?
Am I supposed to use `PooledPostgresqlExtDatabase` or it's overkill and just `PostgresqlExtDatabase` will be fine? Connection string will point to pgbouncer.

They are somewhat orthogonal, and also dependent upon what kind of pooling you use in pgbouncer (session, txn or statement). Peewee's pooled variants will maintain open connections to postgres so doubling-up and using `PooledPostgresqlDatabase` + pgbouncer is probably unnecessary. It also kinda depends on what latency you anticipate with opening the connection (whether to pg itself or to your pgbouncer). There "right" answer is: it depends.

If you use pgbouncer, though, a good starting place is just to use `PostgresqlDatabase` and point it at pgbouncer.

If you want to let peewee handle a connection pool, use the Pooled- variants.
Issue: query.first() mutates the query
Author: klen
State: closed
Created: 2023-02-09 08:32:19

The `query.first()` method mutates the query and it may lead to unexpected behaviour

```python
query = SomeModel.select()
first_item = query.first()
for item in query:
    # This cycle only runs once, because after first() the query is mutated and now it has limit 1
```

The PR fixes the behaviour.

The `first()` method is a bit of a special-case. It applies a limit of 1 to avoid making the server generate the full result-set, as it is assumed that after calling `first()` the query itself will not be iterated. Similarly, the mutation is done in-place so that multiple calls to `first()` do not result in additional queries.

I will add some documentation explaining this.
Issue: excuse, how could i use a variable as table's field
Author: EasyArch-lyy
State: closed
Created: 2023-02-08 09:36:58

eg: 
create table test
(
	id int auto_increment primary key,
	name  varchar(255) null,
);

field = 'name'
query_list = list(test.select().where(test.field=='cc').dicts())

AttributeError: type object 'test' has no attribute 'field'

want to know the api

Post stuff like this on stackoverflow in the future. You use `getattr(test, field) == 'cc'`.
Issue: MySQL introspector returns columns in non-deterministic order
Author: linhr
State: closed
Created: 2023-02-07 08:52:16

When I use the introspector to generate model classes for MySQL tables, the order of columns seems non-deterministic. It turns out that `MySQLDatabase.get_columns()` does not fetch columns ordered by their ordinal position. (Sometimes the columns are simply returned alphabetically, from what I observed.) Could we add `ORDER BY ordinal_position` when selecting from the `information_schema.columns` table in MySQL?

Here is the code for MySQL, and there is no `ORDER BY` clause:
https://github.com/coleifer/peewee/blob/e4f5df28e1b3716de1c006652371744da3743018/peewee.py#L4193-L4197

We can see that for PostgreSQL, columns are returned in a deterministic order:
https://github.com/coleifer/peewee/blob/e4f5df28e1b3716de1c006652371744da3743018/peewee.py#L3983-L3988
Issue: migrator may conflict with AutoIncrementField in sqlite
Author: js515990
State: closed
Created: 2023-01-30 07:38:45

I have created a sqlite table 'user' using peewee:

The table generally looks like this:

> id name
> 1  name1
> 2  name2
> 3  name3
> 4  name4 (deleted)

For column id, i'm using `AutoIncrementField` from `playhouse.sqlite_ext` to prevent the deleted id from being reused

Since the name4 line has been deleted, the id of next created user should be 5

Then i used a migrator to remove the not-null constraint on field name

`migrator = SqliteMigrator(database)`
`migrator.drop_not_null('user', 'name')`

But after this i created a user and the id field was 4, the deleted id was still reused

Without `migrator.drop_not_null('user', 'name')`, the id column works as expected


The problem is that Sqlite does not support a way to modify constraints. So in order to do this, we have to create a new table without the changed constraint, copy data into it, drop the old table, then rename the new table -> old table. In this process, your autoincrement sequence gets lost.
Issue: Value not saved when an alias (column_name) used
Author: ehcpdeveloper
State: closed
Created: 2023-01-25 17:37:28

```
>>> class Test(BaseModel):
...     test1 = DecimalField(constraints=[SQL("DEFAULT 0.00")], null=True)
...     class Meta:
...         table_name = 'test'
...
>>>
>>>
>>> Test.create_table()
>>> Test.insert(test1=12).execute()
1
>>> a=Test[1]
>>> a.test1=20;
>>> a.save()
1
>>> a=Test[1]
>>> a.test1
Decimal('20.00000')
```

Up to now, normal, correct. 

Lets define an alias (2nd field) for test1 field:

```
class Test(BaseModel):
    test1 = DecimalField(constraints=[SQL("DEFAULT 0.00")], null=True)
    test2 = DecimalField(constraints=[SQL("DEFAULT 0.00")], null=True, column_name='test1')
    class Meta:
        table_name = 'test'

>>> a=Test[1]
>>> a.test1=30
>>> a.save()
0
>>>
```

So, that entity unchanged. 
I am not sure if this is an intended behaviour, but, it is obviously misleading. 
I encountered this during dealing with a bug and  this was the issue. it was not saving the value. 



Adding a second field like that is most definitely "undefined behavior" and not advisable. If you want something like this (for some reason), it should be done in Python as perhaps a property. Your example doesn't make clear what the intended goal you were going for, but let's say you wanted something like a normalized accessor to some email field:

```python
class User(Model):
    email = TextField()

    @property
    def normalized_email(self):
        return self.email.lower()
```

Also check out hybrid properties, which provide a similar abstraction that can work at the SQL level as well. Docs: http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#hybrid

Above example:

```python
class User(Model):
    email = TextField()

    @hybrid_property
    def normalized_email(self):
        return self.email.lower()

    @normalized_email.expression
    def normalized_email(cls):
        return fn.LOWER(cls.email)

# Allows things like:
query = User.select().where(User.normalized_email == 'foo@example.com')
# Corresponding to, e.g.:
# select * from user where lower(email) = 'foo@example.com'
```

Intended goal was to use two different field names for same database field.
If this has some kind of "undefined behaviour", then, this should be either forbidden (like with error), or maybe at least warned somehow. 
It does select() etc, but saving does not work and the cause is hard to determine/notice.

"undefined behaviour"s should be avoided as much as possible. 
Anyway, thanks for this nice ORM library/utility.

>Intended goal was to use two different field names for same database field.

To what end? I guess what I'm saying is it sounds like you have a kludge and are unable or unwilling to let go of it.

I'm just trying to provide you feedback, so that you can improve your product/software. Nothing else.
If I have done wrong by providing you some feedback and reporting an "unexpected behaviour", then, sorry !!
Issue: [Feature Request] Event hooks for peewee
Author: Zheaoli
State: closed
Created: 2023-01-23 10:00:47

For now, I'm looking for event hooks in peewee, but I think we don't support event hooks for peewee yet(I have already read the docs, please feel free to correct me if I'm wrong)

I think it's an excellent choice to support event hooks like Django ORM or sqlalchemy like below.

```python
from peewee import event

event.listen(client, "before_cursor_execute", callback_func)
```

The code above means that the callback function will be executed before the final SQL start to be executed.

It's a good feature for a lot of people. For example, we can inject the trace ID in SQL by using SQL comment using https://google.github.io/sqlcommenter/python/sqlalchemy/

FYI https://docs.sqlalchemy.org/en/14/core/events.html

peewee already have signal support see more in https://docs.peewee-orm.com/en/latest/peewee/playhouse.html#signals

Does this feature meets your needs?
 

@Zheaoli - I'm going to pass, as the generally-supported approach for Peewee users has been to override the `Database.execute_sql()` method in these types of cases (where you want before or after a query execution), for example. You can extend this to other core methods like `commit()`, etc.