akumoth/peewee-issues huggingface dataset:
Issue: Fix ModelSelect.get() documentation
Author: ilyakamens
State: closed
Created: 2020-11-18 17:48:21

```
In [4]: (Transaction.select().where(Transaction.id == 999999)).get()
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
/usr/local/lib/python3.7/site-packages/peewee.py in get(self, database)
   6806         try:
-> 6807             return clone.execute(database)[0]
   6808         except IndexError:

/usr/local/lib/python3.7/site-packages/peewee.py in __getitem__(self, item)
   4225             self.fill_cache(item if item > 0 else 0)
-> 4226             return self.row_cache[item]
   4227         else:

IndexError: list index out of range

During handling of the above exception, another exception occurred:

TransactionDoesNotExist                   Traceback (most recent call last)
<ipython-input-4-23b2357d750a> in <module>
----> 1 (Transaction.select().where(Transaction.id == 999999)).get()

/usr/local/lib/python3.7/site-packages/peewee.py in get(self, database)
   6810             raise self.model.DoesNotExist('%s instance matching query does '
   6811                                           'not exist:\nSQL: %s\nParams: %s' %
-> 6812                                           (clone.model, sql, params))
   6813
   6814     @Node.copy

TransactionDoesNotExist: <Model: Transaction> instance matching query does not exist:
SQL: SELECT "t1"."id", "t1"."type", "t1"."account_prn", "t1"."cad", "t1"."amount_cents", "t1"."denied_amount_cents", "t1"."event_id", "t1"."network", "t1"."ach_trans_id", "t1"."open_to_buy_cents", "t1"."auth_txn_id", "t1"."activity_type", "t1"."auth_type", "t1"."description", "t1"."mcc", "t1"."passed_fraud_rules", "t1"."fraud_rules_results", "t1"."timestamp_at", "t1"."raw", "t1"."created_at" FROM "transaction" AS "t1" WHERE ("t1"."id" = %s) LIMIT %s OFFSET %s
Params: [999999, 1, 0]
```

http://docs.peewee-orm.com/en/latest/peewee/api.html#Model.get
Issue: help me BinaryField query 
Author: cbingos
State: closed
Created: 2020-11-18 11:50:06

Sorry to help here:
How to query the “gdmc“ field in List  ?

**`sdgdList =  BinaryJSONField(verbose_name="sdgdList",null=True,)`**

**`sdgdList   List
`**
`
[{"x": "1", "cgs": "11.00", "gbxz": "A", "gdmc": "aaa", "zjqk": "0.48", "zzgs": "0.18", "zjqkCss": "red", "gdblockid": "10",}, {"x": "2", "cgs": "10.06", "gbxz": "A", "gdmc": "bbb", "zjqk": "0.32", "zzgs": "0.17", "zjqkCss": "red", "gdblockid": "10", }]
`
How to query the “gdmc“ field?
`
ModalXXX.sdgdList.contained_by({'gdmc':name})  result None`




Questions belong on stack overflow or you can ask in IRC. This is covered here: http://docs.peewee-orm.com/en/latest/peewee/contributing.html#questions

I'm not sure what you mean by "query the gdmc field", so I'm guessing you mean test it's value?

```python

# test if the "gdmc" field equals some value.
ModelX.select().where(ModelX.sdgdList['gdmc'] == 'some value')
```
Issue: Support "on_conflict" in bulk_create
Author: albatros69
State: closed
Created: 2020-11-11 10:37:20

Hello,

I was trying to take advantage of `bulk_create` (and its batch-support) to insert a lot of new objects. However, unlike `insert_many`, I didn't find a way to specify the `on_conflict` behavior to handle duplicate key errors.
Would this be a possible feature to add?

This came up recently and it is not something I plan to support. See:

https://github.com/coleifer/peewee/issues/2275

OK, thanks anyway for the answer and support and sorry to have overlooked the existing issue (only looked for the opened ones, I admit).


No problem. Like I said in the last issue, the problem is we have no way to map the "successful" IDs to the corresponding instances because of how postgres returns the data.
Issue: Select by tuple in values
Author: ak4nv
State: closed
Created: 2020-11-11 10:19:26

Hi there!
I haven't found in the docs (maybe missed, sorry). Is it possible to make a query like this:

```sql
select * from table where (field1, field2) in (('a', 'b'), ('a', 'c'), ('b', 'c'));
```
Thanks a lot for the greatest library!
p.s. Waiting for `asyncio` support out of the box, don't suggest please this [project](https://github.com/05bit/peewee-async) :smile: 

See: http://docs.peewee-orm.com/en/latest/peewee/query_operators.html#row-values
Issue: The chain of joins with None fields
Author: andr-04
State: closed
Created: 2020-11-10 18:07:36

I've found unexpected behavior due to [this](https://github.com/coleifer/peewee/commit/6bdcd7dc3800654ba034d03b3e7dc01971388e34#r44050928). In case of join to a base model, joined model (but absent; through `LEFT JOIN`) to the field will not be None unless it is the last in the join chain. Models with all fields equal to None is not None originally and requires an additional check for the primary key field to None. It's especially dangerous if you add a new join to the query just, but have to review all places where it is used to change the way of checking to None.

Yes, this was a design decision. If you omit the model how will you traverse the graph?

I can't continue traversing the graph if some _node_ will have all fields equal to None. But it gives the wrong info that the node (model) _exists_. So, this decision doesn't give benefits.

For traversing I use the following helper function:
```
def get_deep(obj_dic, *keys):
    if not keys:
        return obj_dic
    if obj_dic is None:
        return None
    if isinstance(obj_dic, dict):
        return get_deep(obj_dic.get(keys[0], None), *keys[1:])
    else:
        return get_deep(getattr(obj_dic, keys[0], None), *keys[1:])
```

Ok, I follow your point. Not all fields are None, but the chain fields are not None. Anyway, it requires different ways to check the existence of an intermediate and of the last node.

What are your views on adding the last model to the chain as well? It will give the ability to add a helper method that will answer: does the model exists or not? It will also be universal for all nodes in the chain of joins.
Issue: get_id() method returns the foreign key instead of the primary key of model instance
Author: theosotr
State: closed
Created: 2020-11-10 14:12:45

I have the following query (simplified for opening the issue)

``` python
B_al = B.alias()
ret = A.select(B_al.id, A.id)
ret = ret.switch(A).join(B).objects()
for r in ret:
    print(r.get_id()) 
```

Unfortunately, the method `get_id()` unexpectedly returns the foreign key of the table (i.e., B.id) instead of the primary key (i.e., A.id).



You're joining on `B` but you're selecting from `B_al`. The following works just fine for me:

```python
from peewee import *

db = SqliteDatabase(':memory:')
class Base(Model):
    class Meta:
        database = db

class A(Base):
    a = IntegerField()

class B(Base):
    b = IntegerField()
    a = ForeignKeyField(A)

db.create_tables([A, B])

a1 = A.create(a=1)
b1_1 = B.create(id=100, a=a1, b=11)
b1_2 = B.create(id=101, a=a1, b=12)

B_a = B.alias()
q = A.select(B_a.id, A.id).join(B_a)
for row in q:
    print(row.get_id(), row.b.id)

# outputs
# 1 100
# 1 101
```

I had a typo on my previous text. I meant `join(B_al)` instead of `join(B)`.
The method `get_id()` still returns the foreign key of the model instance (tested on sqlite).
peewee version: 3.13.3

I put the corrected query once again here

```python
B_al = B.alias()
ret = A.select(B_al.id, A.id)
ret = ret.switch(A).join(B_al).objects()
for r in ret:
    print(r.get_id()) 
````


You are explicitly asking peewee to patch all attributes onto a single instance by calling `.objects()`, so this is completely expected. Your cursor has 2 values, eached named "id" (for A and B). When you call ".objects()" you're telling peewee to just put all cursor values directly onto a single instance, so you're overwriting A's "id" with the "id" for B.

Solutions:

* do not use .objects()
* alias B's id to some other attribute

Ah, I see.
Thank you!!
Issue: bulk_update throws error when setting array field to None
Author: bdoms
State: closed
Created: 2020-11-05 17:31:55

Here's the situation: I have a model that has an array field where `null=True`. Something like this:

```python
class User(Model):
    emails = ArrayField(CharField, null=True)
```

That's all good, but when I do this:

```python
user = User.create(emails=None) # or get it from a query, etc.

User.bulk_update([user], fields=['emails'])
```

It throws this error:

```
Traceback (most recent call last):
  File ".../peewee.py", line 3099, in execute_sql
    cursor.execute(sql, params or ())
psycopg2.errors.DatatypeMismatch: column "emails" is of type character varying[] but expression is of type text
LINE 1: ... "emails" = CASE "cont...
                                                             ^
HINT:  You will need to rewrite or cast the expression.
```

I investigated and found the SQL it generates looks like this:

```
('UPDATE "user" SET "emails" = CASE "user"."id" WHEN %s THEN %s END WHERE ("user"."id" IN (%s))', [123, None, 123])
```

I believe the issue is that the Python "None" is being cast as a string in the generated SQL via the "%s" instead of properly being converted to NULL.

Luckily the amount of rows I'm dealing with in this particular case is small so I can work around this by just saving each individually, but I can imagine other situations where it could really impact performance.

psycopg2 uses "%s" for all parameter placeholders, and automatically converts `None` to `NULL` so I do not believe this is the issue. To be honest this one has me stumped, as entering the following into the postgres shell also fails with the same error:

```sql
update users 
set emails = case users.id when 1 then null end
where users.id = 1
```

The issue is that postgres wants us to explicitly cast the result of `case` expression:

```sql
-- this works
update "users" set emails = cast(
  case "users"."id" when 1 then NULL end
  as character varying[])
where ("users"."id" in (1));
```

I think I'm just going to have to mark this for now as wontfix since it works fine when you specify an actual list of names. I won't go so far as to say this is a postgres bug, but it certainly feels like one as it works fine with an actual array of emails and only throws an error if you pass an (equally-valid) NULL value.

Thanks for the helpful report. I will update the issue if anything changes.

Thanks for the insight. You're right. It appears to be known behavior for Postgres, see for example: https://dba.stackexchange.com/a/228058

I still think it's something where Peewee or Postgres or maybe even psycopg2 could improve the user experience. I'm just not sure where the best place to ask for an improvement is.
Issue: 'Expression' object has no attribute 'seconds'
Author: ahvahsky2008
State: closed
Created: 2020-11-05 10:40:01

I try get data with creation time more 60 mins.  But its show error 
```
 hour = datetime.datetime.now() - datetime.timedelta(minutes = 60)
 zayavkis =Zayavki.select().where(Zayavki.req_data-hour).seconds/60>60)
```

You're overthinking this. It is quite simple:

```python

cutoff = datetime.datetime.now() - datetime.timedelta(minutes=60)
query = Zayavki.select().where(Zayavki.req_data >= cutoff)
```

As to why this doesn't work, you cannot access the ".seconds" attribute on a peewee expression, as the error clearly states. This is because it's not a `timedelta` object, but rather a representation of a SQL expression.
Issue: Peewee constantly reruns select count in PyCharm REPL
Author: dominusmi
State: closed
Created: 2020-11-04 13:27:33

In the pycharm REPL, once a database model is imported, it will keep running a select count after each instruction execution.
Putting the logger on debug mode, this message appears for each model loaded:

```DEBUG::peewee.py:3087::('SELECT COUNT(1) FROM (SELECT 1 FROM `Model` AS `t1`) AS `_wrapped`', [])```

The problem is that my model are very large database, and so the REPL becomes unuseable since after each instruction it stays stuck for a few seconds.
Unfortunately the line number in the log line seems to be wrong (no idea why) but it points to a line with no logging: 
https://github.com/coleifer/peewee/blob/92494cd9ed01b12826b2fddb25cde221ea34f155/peewee.py#L3087
However, since the only line with `logger.debug` in `peewee.py` is line 3117 (inside `execute_sql`), I would imagine it's an implicit call leading there.

I'm not sure if this is a bug or if it's a required decision, but I wouldn't really understand the necessity in the latter case.
Running peewee version `3.13.3`
The same issue doesn't seem to happen in ipython opened from terminal.
Do you have any idea / help for what's going on? 


Pycharm is probably calling all kinds of stuff. Specifically it's probably evaluating the truthiness or len(). Definitely not a peewee bug. I'd take it up with the pycharm folks.

Using a regular python repl or ipython has no problems at all.

Alright thanks for the feedback!
Issue: peewee AttributeError 'datetime.datetime' object has no attribute 'get_sort_key'
Author: hungter007
State: closed
Created: 2020-10-27 09:57:58

peewee version: 3.13.3
python version: py3.8

error：
![image](https://user-images.githubusercontent.com/32872324/97285859-b9ac7d80-187d-11eb-93aa-1bafec0024ca.png)

model:
    add_dt = peewee.DateTimeField(null=False, default=datetime_now, column_name="c_add_dt")

I don't have enough information to help debug this. Can you provide the query?

('INSERT INTO `t_group` (`c_id`, `c_group_name`, `c_group_code`, `c_doc_ids`, `c_company_id`, `c_is_delete`, `c_add_by`, `c_add_dt`, `c_update_dt`) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s)', ['185ede639b654695869c87b28d31b28d', 'data', '02', '', 'cdf49474297c4454aa2d410ba5eacf60', False, 'user_id', datetime.datetime(2020, 10, 27, 19, 2, 35, 675031), datetime.datetime(2020, 10, 27, 19, 2, 35, 675045)])


when running insert_many get error.

await db_manager.execute(
     Group.insert_many(group_list)
)


This looks like possibly a issue with peewee async? If you can provide a reproducible test that fails I will look into this some more, but `insert_many()` and all this is covered by existing (passing) tests, so I don't think there is anything fundamentally broken here.
Issue: Database(autoconnect=True) is not thread-safe
Author: notEvil
State: closed
Created: 2020-10-26 09:47:58

In `Database.cursor`, `self.connect()` may be attempted by multiple threads and even though the first one will create the connection, all others will get an `OperationalError` due to `connect(reuse_if_open=False)`.

I'm not familiar enough with the code to say that this is the only place where `reuse_if_open` should be `True`.

By default peewee databases use a connection-per-thread.
Issue: peewee.ImproperlyConfigured: database attribute does not appear to be set on the model: <Model: Branch>
Author: medoosh79
State: closed
Created: 2020-10-25 06:26:38


from peewee import *
db = MySQLDatabase('library',user='root', password= '', host='localhost', port=3306 )





class Branch(Model):
    name = CharField()
    code = CharField(null=True,unique=True)
    location = CharField(null=True)




db.connect()

db.create_tables([Branch])


how to solve this issue?

Traceback (most recent call last):
  File "C:/Users/Mohammed/OneDrive - Office 365/Python Projects/Library/test.py", line 20, in <module>
    db.create_tables([Branch])
  File "C:\Users\Mohammed\AppData\Local\Programs\Python\Python38\lib\site-packages\peewee.py", line 3286, in create_tables
    model.create_table(**options)
  File "C:\Users\Mohammed\AppData\Local\Programs\Python\Python38\lib\site-packages\peewee.py", line 6590, in create_table
    if safe and not cls._schema.database.safe_create_index \
  File "C:\Users\Mohammed\AppData\Local\Programs\Python\Python38\lib\site-packages\peewee.py", line 5520, in database
    raise ImproperlyConfigured('database attribute does not appear to '
peewee.ImproperlyConfigured: database attribute does not appear to be set on the model: <Model: Branch>

First off, please take the time to format your code properly. GitHub has links and controls in the editor that will show you how to do this.

Second, please see the documentation. This is covered in all the basic examples.

http://docs.peewee-orm.com/en/latest/peewee/quickstart.html#model-definition

```python
class SomeModel(Model):
    ...
    class Meta:
        database = db_instance
```
Issue: Use link_shared_object instead of link_executable in SQLite detection
Author: sboisson
State: closed
Created: 2020-10-22 16:27:19

`link_executable` doesn't use environment variable `LDFLAGS`, only environment variable `CC` (which usually only contains the name of the compiler, not the linking options)… which can be problematic and cause false negative on setups where SQLite is not installed in system path, but on a custom path.

Also the setup is not attempting to build an executable, but shared objects.


Excellent, thank you!
Issue: foreign key with "_id" in the migrate.add_index
Author: alpo
State: closed
Created: 2020-10-19 14:41:04

I have a foreign key in the ORM object and in the database with name ending with '_id'. Like this:
```
class Thing(BaseThing):
    master_id = ForeignKeyField(AnotherThing, backref='details', index=True)
    other_field =  BigIntegerField(index=True)
    ...
```

and I'm trying to create an index in migration using add_index

Attempt 1
```
def migrate(migrator, database, fake=False, **kwargs):
    migrator.add_index('thing', 'master', 'other_field', unique=True)
```
fails with message: `peewee.InternalError: (1072, "Key column 'master' doesn't exist in table")`

Attempt 2
```
def migrate(migrator, database, fake=False, **kwargs):
    migrator.add_index('thing', 'master_id', 'other_field', unique=True)
```
fails with message: `peewee.InternalError: (1072, "Key column 'master_id_id' doesn't exist in table")`

Is it possible to make a `migrate.add_index` work in this situation?


I cannot replicate this problem:

```python

from peewee import *

db = SqliteDatabase(':memory:')

class Base(Model):
    class Meta:
        database = db

class User(Base):
    username = TextField()

class Tweet(Base):
    user_id = ForeignKeyField(User, backref='tweets')

db.create_tables([User, Tweet])
db.execute_sql('drop index "tweet_user_id"')

from playhouse.migrate import *
migrator = SchemaMigrator.from_database(db)
migrate(
    migrator.add_index('tweet', ('user_id',), unique=True)
)
```

The SQL being generated is correct in the above example:

```sql
CREATE TABLE IF NOT EXISTS "user" ("id" INTEGER NOT NULL PRIMARY KEY, "username" TEXT NOT NULL)
CREATE TABLE IF NOT EXISTS "tweet" ("id" INTEGER NOT NULL PRIMARY KEY, "user_id" INTEGER NOT NULL, FOREIGN KEY ("user_id") REFERENCES "user" ("id"))
-- this is added automatically by peewee --
CREATE INDEX IF NOT EXISTS "tweet_user_id" ON "tweet" ("user_id")
-- we'll drop the automatically-added index explicitly --
drop index "tweet_user_id"

-- this is generated by the migrator --
CREATE UNIQUE INDEX "tweet_user_id" ON "tweet" ("user_id")
```

Also, for what it's worth, using "master_id" is a bit of an anti-pattern, since the field accessor returns an `AnotherThing` instance and not a numeric/unique ID.
Issue: JSON/dict integer keys getting converted to strings when read.
Author: pydatasci-repos
State: closed
Created: 2020-10-16 11:34:03

Here, I have provided a dictionary with integer-based keys `{0:_,1:_,2:_}` to a `JSONField()` attribute named 'folds'.

When I create the object and return it, the folds attribute still serializes with integer keys as expected.

However, when I fetch the object using `get_by_id()`, the `folds.keys()` are now strings. 

Is this expected behavior? Maybe a json.loads or python dict problem rather than a peewee problem, but I just wanted to report it so it's known.


![image](https://user-images.githubusercontent.com/71051965/96254397-3ccafb00-0f83-11eb-81d4-006f75aa70b8.png)

Just going to read it with `folds = {int(k):v for k,v in folds.items()}` when I need that attribute.

Makes me think of this:
![image](https://user-images.githubusercontent.com/71051965/96255136-6a647400-0f84-11eb-9590-7d7185976284.png)

What database are you using? Postgres, Sqlite or MySQL?

@coleifer Sqlite
```
from playhouse.sqlite_ext import SqliteExtDatabase, JSONField
```

I think this comes down to sqlite interpreting object keys as strings. I'd suggest following it up on the sqlite forums if you have more questions on it. https://sqlite.org/forum/forummain

```
sqlite> select json_valid('{1: "k1"}');
0
sqlite> select json_valid('{"1": "k1"}');
1
```

@coleifer Thanks for taking a look. Your `json_valid` test got me on the right track. There are several posts out there pointing to the JSON spec saying that keys must be strings. E.g. https://stackoverflow.com/a/8758771/5739514

Let's see what the sqlite folks say. Surprised it allows it to be stored.
https://sqlite.org/forum/forumpost/28863dcedf

Taking a look at MongoDB, they use a lot of this pattern, but it results in `TypeError: unhashable type: 'list'`. Will keep thinking on a workaround apart from another rel/table.
```
{[
    {
        "_id": 1,
        "attribute1": "beep"
    },
    {
        "_id": 2,
        "attribute2": "boop"
    }
]}
```

Workaround for future humans
```
folds = {"folds": [{"id":1, "name":"a", "other":"beep"}, {"id":2, "name":"b", "other:"boop"}]}
# or
folds = {"folds": [{"name":"a", "other":"beep"}, {"name":"b", "other":"boop"}]}
```
Kinda stinks but at least the inner list can be accessed via index in python. `Foldset.folds["folds"][0]`
Issue: peewee.ModelBase has no attribute 'bulk_update'
Author: instantrad
State: closed
Created: 2020-10-12 08:23:59

As you can see below, when l print type of the peewee model, l get **<class 'peewee.ModelBase'>**
However,  it complains bulk_update does not exist. Also running dir on the Model class, does not contain the **bulk_update**

[2020-10-12 08:20:36,241] {test.py:250} INFO - <class 'peewee.ModelBase'>
[2020-10-12 08:30:35,533] {test.py:251} INFO - ['DoesNotExist', '__class__', '__data__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__rel__', '__repr__', '__setattr__', '__sizeof__', '__sql__', '__str__', '__subclasshook__', '__weakref__', '_as_dict', '_coerce', '_meta', '_normalize_data', '_pk', '_pk_expr', '_populate_unsaved_relations', '_prune_fields', '_schema', "'bulk_create'']
[2020-10-12 08:30:35,533] {taskinstance.py:1150} ERROR - type object 'Study' has no attribute 'bulk_update'
[2020-10-12 08:20:36,241] {test.py:1150} ERROR - type object 'Study' has no attribute 'bulk_update'

What could be the issue?

```python
In [1]: from peewee import *                                                                    

In [2]: class MyModel(Model): 
   ...:     pass 
   ...:                                                                                         

In [3]: MyModel.bulk_update                                                                     
Out[3]: <bound method Model.bulk_update of <Model: MyModel>>
```