akumoth/peewee-issues huggingface dataset:
Issue: JSONField.concat always uses json.dumps on parameters
Author: crvv
State: closed
Created: 2020-03-17 16:48:39

```Python
from peewee import Model, PostgresqlDatabase
from playhouse.postgres_ext import BinaryJSONField

database = PostgresqlDatabase(database='postgres', autoconnect=True)

class Test(Model):
    x = BinaryJSONField(index=False)
    y = BinaryJSONField(index=False)
    class Meta:
        database = database

q = Test.select(Test.x.concat(Test.y)).dicts()
print(list(q))
```

This raises exception `TypeError: Object of type BinaryJSONField is not JSON serializable`.

Thanks, this is fixed.
Issue: How to use composite type as function argument (PostgresQL)
Author: zmwangx
State: closed
Created: 2020-03-11 12:36:03

PostgresQL supports composite types (e.g. row types) as function arguments, but I can't figure out how to use a model as an argument of a `peewee.fn.func`.

Say we have this schema

```sql
CREATE TABLE IF NOT EXISTS post (
    id INTEGER NOT NULL PRIMARY KEY,
    votes INTEGER NOT NULL,
    posted TIMESTAMPTZ NOT NULL DEFAULT CURRENT_TIMESTAMP
);
```

and a function

```sql
CREATE OR REPLACE FUNCTION score(post)
	RETURNS double precision
	AS $$ SELECT $1.votes / EXTRACT(EPOCH FROM now() - $1.posted); $$
	LANGUAGE SQL
	STABLE;
```

we want to build this query:

```sql
SELECT id, score(post) FROM post;
```

How do we approach this?

I cannot use the model `Post` as an argument to `peewee.fn.score`:

```py
Post.select(Post.id, fn.score(Post))
# => peewee.ProgrammingError: can't adapt type 'ModelBase'
```

even `SQL` doesn't work unless I force an alias on the model:

```py
Post.select(Post.id, SQL("score(post)"))
# peewee.ProgrammingError: column "post" does not exist (because 'FROM "post" AS "t1"' in the constructed query)
```

```py
Post_ = Post.alias("post")
Post_.select(Post_.id, SQL("score(post)"))
# This one works.
```

I'm currently using the workaround above. Is there any better way to achieve this? Sorry if I missed something in the docs, I think I scoured every corner. (And sorry for the barrage of issues I've opened recently.)

I believe this is fixed, thanks for the report.

It works now, thanks.
Issue: Clarify PostgresqlMigrator.set_search_path in docs
Author: albireox
State: closed
Created: 2020-03-10 19:49:14

It may be beneficial to make a bit more clear that `PostgresqlMigrator.set_search_path` needs to be added to `migrate` for it to take effect. That may be obvious in hindsight but because of the way it's presented in the docs API it took me a good while to realise that. An example in the main section of the Schema Migration docs would be useful.

Can you give an example of what you mean? I would have thought that the setting of `schema` on models would have been adequate, and that the migration methods would take the schema into consideration, though I may be wrong.

I may be misunderstanding how it's supposed to work but I'd guess that if I do

```python
 selected_field = peewee.BooleanField(default=True, null=False)
 migrator = PostgresqlMigrator(database)
 migrate(migrator.add_column('temp_guide', 'selected', selected_field))
```

and the `temp_guide` table is in a schema called `sandbox` that won't work because at any point I'm passing the `TempGuide` model to the migrator. Instead, doing

```python
 selected_field = peewee.BooleanField(default=True, null=False)
 migrator = PostgresqlMigrator(database)
 migrate(
    migrator.set_search_path('sandbox'),
    migrator.add_column('temp_guide', 'selected', selected_field)
)
```

works fine. But it took me a bit to realise that `migrator.set_search_path('sandbox')` had to be inside the `migrate` function.

Not saying that there's anything wrong per se in the docs, but it could be made a bit clearer.

Thanks, added a note.
Issue: Python descriptor protocol support for correct attribute injection
Author: dimma837
State: closed
Created: 2020-03-10 10:37:42

Hi,

I use peewee ORM with python [dependency injection framework](https://github.com/ivankorobkov/python-inject) for database instance injection. But peewee does not implement Python descriptor protocol on class attributes access, so it does not work in conjunction with DI.

I described this problem including code examples and exception traceback to the author of the DI framework, and he said that it's peewee concern.

ivankorobkov/python-inject#49

Can you implement it? Or maybe there are other workarounds?
Issue: Improve reflection for AutoField/IntegerField
Author: ilyakamens
State: closed
Created: 2020-03-05 14:08:38

Specifically, don't assume `IntegerField(primary_key=True)` is an `AutoField()` if the database isn't SQLite.

See https://github.com/coleifer/peewee/issues/2124 for more context.

I'm going to pass on this for now, but may reconsider it at some point.
Issue: prefetch fails when foreign key field or target primary key field not selected
Author: zmwangx
State: closed
Created: 2020-03-04 12:29:31

This is not a bug, but rather a minor UX issue. Consider this simple example:

```python
import peewee


db = peewee.SqliteDatabase(":memory:")


class _BaseModel(peewee.Model):
    class Meta:
        database = db


class User(_BaseModel):
    name = peewee.TextField()


class Entry(_BaseModel):
    user = peewee.ForeignKeyField(User, backref="entries")
    content = peewee.TextField()


def main():
    db.create_tables([User, Entry])
    User.insert(name="foo").execute()
    Entry.insert(user_id=1, content="bar").execute()
    user_query = User.select(User.name)  # User.id not selected
    entry_query = Entry.select(Entry.content)  # Entry.user_id not selected
    peewee.prefetch(user_query, entry_query)


if __name__ == "__main__":
    main()
```

The prefetch fails when either `User.id` or `Entry.user_id` is not selected, as prefetch can't link the models together without either. The errors are slightly cryptic too if one doesn't realize what's going on

```python-traceback
...
  File "/Users/zmwang/.pyenv/versions/3.8.0/lib/python3.8/site-packages/peewee.py", line 7421, in store_instance
    identity = field.rel_field.python_value(instance.__data__[attname])
KeyError: 'user'
```

```python-traceback
...
  File "/Users/zmwang/.pyenv/versions/3.8.0/lib/python3.8/site-packages/peewee.py", line 7412, in populate_instance
    identifier = instance.__data__[field.rel_field.name]
KeyError: 'id'
```

(It may seem silly to specify the fields to be selected in this simple example, but in general I refrain from `SELECT *` and only select what I need.)

Maybe `prefetch` could add the relation fields it require automatically? Not sure about feasibility implementation-wise. Failing that, maybe add a little note to the API docs?

Yeah, prefetch needs those fields to be able to reconstruct the graph. I'll make a note in the docs.
Issue: Unable to use dict pragmas with sqliteq
Author: dev-dsp
State: closed
Created: 2020-03-04 12:13:57

Hi

https://github.com/coleifer/peewee/blob/master/playhouse/sqliteq.py#L228

Here for pragmas being dictionary it fails with 
```
...
    pdict = dict((k.lower(), v) for (k, v) in pragmas)
ValueError: too many values to unpack (expected 2)
```

Not sure how to better fix it in code. Not urgent also, I will just use list of tuples for now.
Issue: Hybrid expression with JOIN
Author: vltr
State: closed
Created: 2020-03-03 14:16:31

Howdy! I'm fairly new to `peewee`, but have some strong background on SQLAlchemy (and all the vices that come with it). I'm trying to create a custom hybrid expression that correlates to a third (or N) table. I'll try to demonstrate in an example (non-tested) code:

```python
class BaseModel(Model):
    class Meta:
        database = database


class Person(BaseModel):
    id = PrimaryKeyField(column_name="person_id")
    name = CharField(max_length=255, column_name="person_name")
    username = CharField(max_length=255, column_name="person_username", null=True)



class PersonTree(BaseModel):
    id = PrimaryKeyField(column_name="person_tree_id")
    name = CharField(max_length=255, column_name="person_tree_name")
    code = CharField(max_length=255, column_name="person_tree_code")

    person = ForeignKeyField(
        column_name="person_id",
        model=Person,
        field="id",
        backref="tree",
    )


class Article(BaseModel):
    id = PrimaryKeyField(column_name="article_id")
    name = CharField(max_length=255, column_name="article_name")

    branch = ForeignKeyField(
        column_name="person_tree_id",
        model=PersonTree,
        field="id",
        backref="articles",
    )

    @hybrid_property
    def username(self):
        """
        This gives me the possibility to grab the direct username of an article
        """
        return self.branch.person.username

    @username.expression
    def username(cls):
        """
        What if I wanted to do: Article.query().where(Article.username == "john_doe") ?
        """
        pass
```

With the `username` `hybrid_property` on `Article`, I can get the `username` of the `Person` related to an `Article` using the `PersonTree` as a correlation, so far so good, but ... What if I wanted to "create a shortcut" to query all `Articles` created by the `"john_doe"` `Person` username, without declaring the `JOIN`s every time I make the query and without `.filter(branch__person__username="john_doe")`? I know it's possible with SA (to a great extent), but I'm finding this hard to accomplish with `peewee`.

Just for example, here's the SQL I hope to be able to construct:

```sql
SELECT
    *
FROM
    article a
    JOIN person_tree pt ON a.person_tree_id = pt.person_tree_id
    JOIN person p ON pt.person_id = p.person_id
WHERE
    p.username = 'john_doe';
```

Thanks a lot in advance!

Questions typically go on stackoverflow.
Issue: Field help_text attribute?
Author: jvickroy
State: closed
Created: 2020-03-03 02:49:44

What is the intent of the help_text attribute?
Looking at the code, it seems to only be be listed in the class Field definition.
I would like to be able to access the help_text strings, in my scripts, but do not know how to do so.
Could someone point in the right direction?
Thanks for your time.
P.S.  peewee is great!

It's available to provide a standard way of adding metadata. `flask-peewee` &co made use of it. To access the help text, just access `field.help_text`. To iterate a model's fields, `ModelClass._meta.sorted_fields` returns the fields in order of declaration.
Issue: How to compare with boolean for PostgreSQL?
Author: EugeneRymarev
State: closed
Created: 2020-03-02 04:09:04

Hello!
`Table.coumn == False` in peewee give me
`Table.column = 0` in sql. 
And i receive error "can't compare boolean and int".

`Table.Column == SQL('false')` works fine, but that wrong way

Huh? I've added a regression test that shows the full range of boolean comparisons. I could not replicate any problems on my machine: a9ebf6bdf2c7a74a802bd6860d1357b091b332a2
Issue: How works count subquery?
Author: EugeneRymarev
State: closed
Created: 2020-03-01 20:20:35

I need count in subquery. Example at http://docs.peewee-orm.com/en/latest/peewee/hacks.html#other-methods
My code:
```
subquery_condition = (UserSurvey.user == User.id) & (UserSurvey.is_current == False)
subquery = (UserSurvey.select(fn.COUNT(UserSurvey.id)).where(subquery_condition))
condition = User.next_notify & (User.next_notify <= dt_now) & (subquery == 0)
```
I receive error:

> Traceback (most recent call last):
>   File "/mnt/d/projects/venvs/survey_bot/lib/python3.8/site-packages/click/core.py", line 717, in main
>     rv = self.invoke(ctx)
>   File "/mnt/d/projects/venvs/survey_bot/lib/python3.8/site-packages/click/core.py", line 1137, in invoke
>     return _process_result(sub_ctx.command.invoke(sub_ctx))
>   File "/mnt/d/projects/venvs/survey_bot/lib/python3.8/site-packages/click/core.py", line 956, in invoke
>     return ctx.invoke(self.callback, **ctx.params)
>   File "/mnt/d/projects/venvs/survey_bot/lib/python3.8/site-packages/click/core.py", line 555, in invoke
>     return callback(*args, **kwargs)
>   File "/mnt/d/projects/survey_bot/project/tlgrm.py", line 37, in notifications
>     dialog_bot.send_notifications()
>   File "/mnt/d/projects/survey_bot/project/telegram_dialog/bot.py", line 280, in send_notifications
>     users = get_user_for_notifications()
>   File "/mnt/d/projects/survey_bot/project/views/user.py", line 103, in get_user_for_notifications
>     condition = User.next_notify & (User.next_notify <= dt_now) & (subquery == 0)
>   File "/mnt/d/projects/venvs/survey_bot/lib/python3.8/site-packages/peewee.py", line 817, in __eq__
>     return self._hash == other._hash
> AttributeError: 'int' object has no attribute '_hash'
>  
> Process finished with exit code 1

Fixed. As a workaround until I tag a new release, you can use:

```python
Expression(subquery, OP.EQ, 0)
```

Instead of `subquery == 0`.
Issue: add jsonb array oid
Author: james-lawrence
State: closed
Created: 2020-02-28 20:56:16

attempts to add JSONB array column to postgresql.

however unsure of how to use it. the generated code looks correct

however these fail:
```python
Model.update(array=[{}]).where(Model.id == 1).execute()
Model.create(array=[{}])
```

I don't think using arrays with jsonb is a sensible thing for anyone to do, since json has a native list type already. Going to pass on this.

whether you think it is sensible is immaterial, postgresql allows it. and clearly some people have usecases for it.
Issue: generate_models on SQLite database throws KeyError when database contains invalid foreign keys
Author: kohavytank
State: closed
Created: 2020-02-23 16:49:03

Because foreign key constraints in SQLite can be disabled (and is disabled by default), it is possible to create a table with an invalid foreign key, like this:
```python
from peewee import SqliteDatabase
sqlite_db = SqliteDatabase(':memory:')
sqlite_db.execute_sql("""CREATE TABLE IF NOT EXISTS "table1" ("_id" INTEGER PRIMARY KEY,"name" TEXT,"value" INTEGER REFERENCES "non-existent-table"("non-existent-column"))""")
```

When running generate_models on such databases a KeyError is thrown.
```
>>> from playhouse.reflection import generate_models
>>> generate_models(sqlite_db)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\kohavy\.virtualenvs\dtu--X-W94Eo\lib\site-packages\playhouse\reflection.py", line 775, in generate_models
    return introspector.generate_models(**options)
  File "C:\Users\kohavy\.virtualenvs\dtu--X-W94Eo\lib\site-packages\playhouse\reflection.py", line 674, in generate_models
    include_views)
  File "C:\Users\kohavy\.virtualenvs\dtu--X-W94Eo\lib\site-packages\playhouse\reflection.py", line 661, in introspect
    related_name=related_names.get(src))
  File "C:\Users\kohavy\.virtualenvs\dtu--X-W94Eo\lib\site-packages\playhouse\reflection.py", line 130, in set_foreign_key
    self.rel_model = model_names[foreign_key.dest_table]
KeyError: 'non-existent-table'
```

Is there a way to generate models for a database without the foreign key constraints? or a way to hook a callback to generate the appropriate field (IntegerField in the above case) when a foreign key is missing?

Ehhh, man that's a really quite badly-broken schema. Not using constraints is one thing, but using constraints and then having them be extremely broken is not really something I intend to support.

If you need a workaround, just use pwiz' "-t" option to explicitly specify the list of un-broken tables, and then enter in the others by hand.
Issue: Update record through foreign key
Author: albireox
State: closed
Created: 2020-02-21 08:56:33

If I have two tables, say `User` and `Tweet` related via a foreign key, and I try to do

```python
tweet = Tweet.select().first()
tweet.user.name = 'John Smith'
tweet.save()
```

The new name for the user is not saved (although no error is raised and `tweet.save()` returns `1`). The same is true if I do `tweet.user.save()`.

Is is possible to update the related record in peewee? I know I can do the query on `User` and join with `Tweet`, but sometimes it'd be convenient to update it through the foreign key.

Peewee is an active record ORM, so you would need to `save()` any instances you modify.
Issue: playhouse.reflection.generate_models does not reflect field constraints
Author: gregspurrier
State: closed
Created: 2020-02-18 21:27:31

Using peewee 3.13.1 with PostgreSQL 9.6.15 and psycopg2 2.8.2, I noticed that a model class generated with `generate_models()` had a column that did not reflect its specified `constraints`:

```python
from peewee import TextField, Check
from playhouse.reflection import generate_models

class TestModel(db.Model):
    a_field = TextField(constraints=[Check("a_field != ''")])

TestModel.create_table()
TestModel.a_field.constraints
#> 
models = generate_models(db.database)
models["testmodel"]._meta.columns["a_field"].constraints
#> None
```

Here is the table definition:

```
# \d testmodel
                             Table "public.testmodel"
 Column  |  Type   | Collation | Nullable |                Default
---------+---------+-----------+----------+---------------------------------------
 id      | integer |           | not null | nextval('testmodel_id_seq'::regclass)
 a_field | text    |           | not null |
Indexes:
    "testmodel_pkey" PRIMARY KEY, btree (id)
Check constraints:
    "testmodel_a_field_check" CHECK (a_field <> ''::text)
```


We don't currently support inspecting constraints besides "NOT NULL" and an implied "UNIQUE".
Issue: playhouse.reflection.generate_models does not reflect index_type
Author: gregspurrier
State: closed
Created: 2020-02-18 21:12:53

Using peewee 3.13.1 with PostgreSQL 9.6.15 and psycopg2 2.8.2, I noticed that a model class generated with generate_models() had a column that did not reflect its specified `index_type`:

```python
# db is a FlaskDB object
class TestModel(db.Model):
    a_field = TextField(index=True, index_type="BRIN")

TestModel.create_table()
models = generate_models(db.database)
models["testmodel"].a_field.index
#> True
models["testmodel"].a_field.index_type
#> None 
```

Here is the table definition:

```
# \d testmodel
                             Table "public.testmodel"
 Column  |  Type   | Collation | Nullable |                Default
---------+---------+-----------+----------+---------------------------------------
 id      | integer |           | not null | nextval('testmodel_id_seq'::regclass)
 a_field | text    |           | not null |
Indexes:
    "testmodel_pkey" PRIMARY KEY, btree (id)
    "testmodel_a_field" brin (a_field)
```

Currently we don't support detection of special index-types.
Issue: Fix tests for UTC >=+ 4 timezones
Author: felixonmars
State: closed
Created: 2020-02-16 19:59:54

Currently test_timestamp_field_from_ts assumes the timezone to be less than +4 so it remains the same day. This would fail when running with for example UTC+8:

```
======================================================================
FAIL: test_timestamp_field_parts (tests.fields.TestTimestampField)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/python-peewee/src/peewee-3.13.1/tests/fields.py", line 1066, in test_timestamp_field_parts
    self.assertEqual(d, 2)
AssertionError: 1 != 2

----------------------------------------------------------------------
Ran 859 tests in 3.450s

FAILED (failures=1, skipped=102)
```

Just comparing it with dt_utc.day fixes the failure.

Thanks!
Issue: playhouse/introspector: doesn't work with capitalized table names (Postgres DB)
Author: fmarcolino
State: closed
Created: 2020-02-16 18:43:53

Hello, I'm trying inspect a table, but he doesn't find her. A exception is raised: UndefinedTable. I think that there is a bug in the query. When we don't put the double quotes in the table name, the postgres database consider name table as lower case.

![image](https://user-images.githubusercontent.com/19367970/74610484-60773200-50d2-11ea-9112-6317b5252cce.png)


I did here fixing the identifier table, in the PostgresqlMetadata Class and he works now.

What is the best solution?

Vlw flw

Thanks, I've merged your patch.
Issue: postgresql: missing support for arrays of uuid's and jsonb
Author: james-lawrence
State: closed
Created: 2020-02-14 15:10:58

see #1470 for related PR.

relevant oids:
UUID array OID: 2951
JSONB array OID: 3807

Merged patch for UUID array.

An array of json doesn't make sense to me, since json supports lists as a fundamental data type.
Issue: Can't convert string to float 
Author: vkachhawa
State: closed
Created: 2020-02-12 06:58:52

I have a database which has the column "loc" which has latitude and longitude data stored in a string:like this " [36.166540711883776,-115.14080871936427]" it's a complete string now i am able to get values of latitude and longitude using query like this: "36.166540711883776".Now how do i convert them to a float value in the query so i can use them in the great circle distance formula My code looks like this:
radius = 10

    query_lat = (new_table.select(fn.substr(new_table.loc, fn.instr(new_table.loc, "[") + 1, fn.instr(new_table.loc, ",") - 2)))
    query_long = (new_table.select(fn.substr(new_table.loc, fn.instr(new_table.loc, ",") + 2,(fn.instr(new_table.loc, "]") - fn.instr(new_table.loc, ",") - 2))))

    output_query = (new_table.select(new_table.id, new_table.loc, new_table.userId, new_table.description, new_table.price, new_table.status).where(
            (fn.acos(
             fn.cos(fn.radians(36.166540711883776)) * fn.cos(
                 query_lat) * fn.cos(fn.radians(-115.14080871936427) - query_long)
             + fn.sin(fn.radians(36.166540711883776)) * fn.sin(query_lat)) * 3969) <= radius))

This is not your personal "help" message board. Use stackoverflow if you need help. Also you contacted me directly via my website in addition to posting here. Please don't spam.

As I said in my email response to you, your data is fucked. You're storing lat/lon in a string field like `[1.234, 2.345]`. Just process your data and separate them out into two separate numeric columns, and your life will be easier *plus* your queries will execute dramatically faster.

`the_string_expression.cast('numeric')` should do it, by the way.
Issue: playhouse.reflection.generate_models reorders compound index keys
Author: gregspurrier
State: closed
Created: 2020-02-11 23:16:07

Using peewee 3.13.1 with PostgreSQL 9.6.15 and psycopg2 2.8.2, I noticed that a model class generated with `generate_models()` had the incorrect order for the keys of a reflected compound index:

```
# \d typicaluser
                              Table "public.typicaluser"
   Column   |  Type   | Collation | Nullable |                 Default
------------+---------+-----------+----------+-----------------------------------------
 id         | integer |           | not null | nextval('typicaluser_id_seq'::regclass)
 first_name | text    |           | not null |
 last_name  | text    |           | not null |
 email      | text    |           | not null |
Indexes:
    "typicaluser_pkey" PRIMARY KEY, btree (id)
    "typicaluser_last_name_first_name" btree (last_name, first_name)
```

vs. 

```
>>> from playhouse.reflection import generate_models
>>> models = generate_models(database)
>>> models["typicaluser"]._meta.indexes
[(['first_name', 'last_name'], False)]
```

Note that the generated model has `['first_name', 'last_name']` vs the `(last_name, first_name)` order in the database.

Verified on Postgres 12 as well. Thanks for the report.

Fixed. Thanks again for the report!