akumoth/peewee-issues huggingface dataset:
Issue: About the postgresql update problem
Author: xiaoqge
State: closed
Created: 2021-05-06 16:15:31

I am using pgsql 13.2, I tried to execute the following code, and I commented the commit, but an error occurred after executing the update, may I ask where the problem is, thank you very much
```
from peewee import *

db = PostgresqlDatabase('peewee_test',user='postgres',host="10.10.17.2",password='11')
db2 = PostgresqlDatabase('peewee_test',user='postgres',host="10.10.17.2",password='11')  # 2nd connection

class Reg(Model):
    name = CharField(null=True,unique=True)
    key = TextField()
    class Meta:
        database = db

class Reg2(Reg):  # model class for accessing table using 2nd conn
    class Meta:
        database = db2
        table_name = 'reg'

class Test():

    def test_db(self):
        db.create_tables([Reg])

        with db.manual_commit() as tx:
            db.begin()
            for i in range(1,10):
                try:
                    Reg.create(key='k1',name="test")  # create a row using first conn
                    print("11111")
                except:
                    Reg.update(key='k2').where(Reg.name == "test22").execute()
                    print("22222")

                db2.connect()  # query table using 2nd conn
                print('is "k1" visible to conn2 BEFORE commit?')
                print('rows in "reg" table: %s' % Reg2.select().count())

                #db.commit()
                print('is "k1" visible to conn2 AFTER commit?')
                print('rows in "reg" table: %s' % Reg2.select().count())

                db2.close()

        #db.drop_tables([Reg])

T=Test()
T.test_db()
```

Ah, I see the problem.

When Postgresql runs into an IntegrityError (the exception that occurs when you violate the unique constraint, e.g.) then you *have* to rollback the transaction. The connection gets into a bad state and no further queries can be run until you rollback. The problem is you have started your Tx before you insert the first row. So when you are forced to rollback, you are rolling back the insert.
Issue: Postgresql transaction problem
Author: xiaoqge
State: closed
Created: 2021-05-06 14:08:51

In the code below, I commented the commit, but it can still be written to the database. I hope to write to the database after the commit. I used mysql before, but it is not normal to migrate to Postgresql. Could you please tell me where do I do this? is it wrong? thank you very much for your help
```

                with db.manual_commit() as tx:
                    db.begin()
                    for b in range(curr_storage_hight, curr_storage_hight + 1000):
                        if b > end_block:
                            print("current wait new block")
                            sleep(20)
                            break
                        self.starttime = datetime.datetime.now()
                        print("Current check_his block scripthash key  %s" % b)
                        self.bnumber = b
                 
                        c =Currnumber.get_or_none(name="gradenumber")
                        if c ==None:
                            Currnumber.create(name="gradenumber", number=self.bnumber)
                        else:
                            Currnumber.update(number=self.bnumber).where(Currnumber.name == "gradenumber").execute()
                    # db.commit()
```
Issue: DataSet "peewee.OperationalError: duplicate column name" raised when unique column name with different case is present in thawed CSV file
Author: ldconejo
State: closed
Created: 2021-05-03 17:01:33

Issue originally reported by @bromleysj

Running peewee-3.14.1 on Python 3.9.

Consider the following example:

```
# demo.py
from playhouse.dataset import DataSet

database = "test.db"

database = DataSet(f"sqlite:///{database}")

users = database["Users"]
users.insert(user_id="test")
users.create_index(["user_id"], unique=True)
users.delete(user_id="test")

users.thaw(filename='test_accounts.csv', format="csv")
```

If we try to add data from a .csv file with a column named "USER_ID", all uppercase, a _peewee.OperationalError: duplicate column name_ exception will be raised. The issue only occurs if the affected column has been marked as unique when creating the table and can be worked around by making the affected column in the .csv file match the case of the corresponding column in the existing table.

This .csv file will fail:

```
USER_ID,EMAIL,NAME,LASTNAME
Larisa.Yesima75,Larisa,Yesima,Larisa.Yesima75@testmail.com
```

This .csv file will pass:

```
user_id,EMAIL,NAME,LASTNAME
Larisa.Yesima75,Larisa,Yesima,Larisa.Yesima75@testmail.com
```

I don't necessarily agree that this needs to be fixed by Peewee -- presumably you can sanitize or normalize this data.
Issue: How can I split the table in Peewee?
Author: nic519
State: closed
Created: 2021-05-02 06:10:29

```
class User(Model):
    id = IntegerField()
    name = CharField()

    Class Meta:
        database = db # db connection
        db_table = 'user_00' # Here is the problem
```

We have a lot of users, and I have to split the table like user_00,user_01,user_0...user_09.

If I use the field db_table, I can't do that, and I also don't want to create more model.

BTW, I know the function called db_table_func, but there are no more usage in the doc.

How do you use the split table in your project?

Thx, I really need help!

You're going to have a hard time, basically.

You might create a VIEW or something that unions together all these tables, and just select from that view for read-only queries.

For writing you will need to either

1. change the `table_name` at runtime (`db_table` is deprecated, it has been renamed to `table_name`)
2. create a model-per-table and pick the right one at runtime
Issue: Added two special expressions, for BooleanField
Author: daveusa31
State: closed
Created: 2021-04-30 18:46:01

To write queries like this,
```python
User.select().where(User.is_activated.is_true())
```
And not like this
```python
User.select().where(User.is_activated == True)
```



You don't even need `.is_true()` honestly, you can just:

```python
.where(User.is_activated)
```

Or for false:

```python
.where(~User.is_activated)
```

Going to pass on this change.
Issue: Postgresql  support TimescaleDB ?
Author: xiaoqge
State: closed
Created: 2021-04-21 04:28:25

Hello, does Postgresql support the extension of TimescaleDB?

I don't know anything about TimescaleDB. Peewee supports anything that speaks the postgres protocol via the psycopg2 driver. If psycopg2 supports it, then its likely supported in peewee.
Issue: Control result iterator in prefetch
Author: surajrav
State: closed
Created: 2021-04-20 08:29:12

Hi, I would like to know if there is a way to specify the iterator per row in a prefetch query.

Ideally,  I would like to add a `.dicts()` to the end of my prefetch query along with specifying which fields to add to the dict that may not be in the base queries select.

*Current Model and Query Setup*

The scenario I have is as follows:

```
class User(BaseModel):
     .... //some user fields
     
class Tool(BaseModel):
    .... //other fields
    location = peewee.ForeignKeyField(Location, backref='tools')

class Location(BaseModel)
    users = peewee.ManyToManyField(User, backref='locations')
    ... // other fields

LocationUser = Location.users.get_through_model()

locations = Location.select(Location.field1, Location.field2, fn.COUNT(Tool.id).alias('tools_count')).join(Tool, JOIN.LEFT_OUTER).group_by(Location)
res = prefetch(locations, LocationUser, User)
```

* Current Result*: List of Location Objects

*Desired Result*
```[{'id': 'idval', 'field1': 'val1, 'tools_count': 0, 'users': [{user dict fields}, ...]} ,...]```

For now, I'm taking the returned list from prefetch and serializing it by running over this list. But I'm afraid that this means 2 iterations over the desired set and want to know if I can supply this serializing logic via a hook or something.

Please let me know if my provided context is insufficient 

Prefetch is not designed as a replacement for serialization - it is to help reduce queries in certain cases. The complexity of mapping relations back to instances precludes providing different result-types (like dicts or tuples) -- so models are always returned. Any additional processing can then work on the model instances which have efficiently been loaded into memory.

>But I'm afraid that this means 2 iterations over the desired set and want to know if I can supply this serializing logic via a hook or something.

I doubt very much this is as much of a problem as you imagine. I'd suggest profiling if you're in doubt.
Issue: bulk_create does not work with bulk created foreignkey
Author: elya5
State: closed
Created: 2021-04-19 09:40:43

Hi,

I have the following models:
```
class WWW(Model):
    mdate = CharField(null=True)
    key = CharField(null=True)
    title = CharField(null=True)
    url = CharField(null=True)

    class Meta:
        database = d

class AuthorField(Model):
    name = CharField()
    www = ForeignKeyField(WWW, backref='names')

    class Meta:
        database = d
```

I have to insert a lot of entries so I wanted to use bulk insertion. However I'm running into the following error (the first part is just to demonstrate that it works without `bulk_create`)
```
>>> a=WWW(title='test')
>>> b=AuthorField(name='testname')
>>> b.www = a
>>> a.save()
1
>>> b.save()
1
>>> a=WWW(title='test2')
>>> b=AuthorField(name='testname2')
>>> b.www = a
>>> WWW.bulk_create([a])
>>> AuthorField.bulk_create([b])
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.9/site-packages/peewee.py", line 3144, in execute_sql
    cursor.execute(sql, params or ())
sqlite3.IntegrityError: NOT NULL constraint failed: authorfield.www_id

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/user/.local/lib/python3.9/site-packages/peewee.py", line 6378, in bulk_create
    res = cls.insert_many(accum, fields=fields).execute()
  File "/home/user/.local/lib/python3.9/site-packages/peewee.py", line 1907, in inner
    return method(self, database, *args, **kwargs)
  File "/home/user/.local/lib/python3.9/site-packages/peewee.py", line 1978, in execute
    return self._execute(database)
  File "/home/user/.local/lib/python3.9/site-packages/peewee.py", line 2745, in _execute
    return super(Insert, self)._execute(database)
  File "/home/user/.local/lib/python3.9/site-packages/peewee.py", line 2474, in _execute
    cursor = database.execute(self)
  File "/home/user/.local/lib/python3.9/site-packages/peewee.py", line 3157, in execute
    return self.execute_sql(sql, params, commit=commit)
  File "/home/user/.local/lib/python3.9/site-packages/peewee.py", line 3151, in execute_sql
    self.commit()
  File "/home/user/.local/lib/python3.9/site-packages/peewee.py", line 2917, in __exit__
    reraise(new_type, new_type(exc_value, *exc_args), traceback)
  File "/home/user/.local/lib/python3.9/site-packages/peewee.py", line 190, in reraise
    raise value.with_traceback(tb)
  File "/home/user/.local/lib/python3.9/site-packages/peewee.py", line 3144, in execute_sql
    cursor.execute(sql, params or ())
peewee.IntegrityError: NOT NULL constraint failed: authorfield.www_id
>>> 
```

It works if I save each of the `WWW`s individually but that is way slower. Is there any way around this?

The bulk create API doesn't set the fk value automatically.

Just wrap it in a transaction and it will be plenty fast.
Issue: Passing max_length to CharField when used in ArrayField
Author: TimothyBramlett
State: closed
Created: 2021-04-19 03:54:39

How would I pass max_length here:

```
some_field = ArrayField(peewee.CharField(max_length=20), null=True)
```

When I try to do that I get:

```
TypeError: 'CharField' object is not callable
```

Please post questions to stackoverflow.

```python
some_field = ArrayField(CharField, field_kwargs={'max_length': 20}, null=True)
```

http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#ArrayField
Issue: Add ability to create an instance from a back reference
Author: mikemill
State: closed
Created: 2021-04-16 20:40:14

Allow an instance of a model to be created using the back reference.  For example, if I have a `User` and I want to create a `Tweet` for that user then allow it to be used as `user.tweets.create()`.

On a practical level this helps a lot when there are logs associated with a user or similar object.  Compare the above with having to import the log model and do `Log.create(user=user, otherparam=value)` which can cause circular import issues with large projects.

The patch provide is intended to provide an example.  I tried to fit the style but am open to any changes needed.

Thanks 

The patch looks fine, but I have tried to stay away from that particular pandora's box. In the simple case it's straightforward, but it breaks the backref abstraction a bit, and there are just a few too many edge-cases. I prefer to keep it as a naive/simple `Select`.

If you really want this in your own app, I've pushed a patch 626b34856eeb842ad27c885f3c1fba95cce78399 that allows you to override the backref accessor class on your foreign-keys - making it hopefully easier to extend and use your own implementation.
Issue: "can't set attribute" using "Recursive CTEs" on Nuke Python
Author: websterek
State: closed
Created: 2021-04-13 20:05:23

Hi!
I have problem will running peewee using Python built into Nuke software:
```
Python version
2.7.16 (v2.7.16-dirty:413a49145e, Aug  6 2019, 14:37:13) [MSC v.1913 64 bit (AMD64)]
Version info.
sys.version_info(major=2, minor=7, micro=16, releaselevel='final', serial=0)
```
In "normal" Python installation everything works as expected, only Python inside Nuke is problematic... for some reason :(

Here is my code raising error, it's nearly one to one copy of example from [peewee documentation:](http://docs.peewee-orm.com/en/latest/peewee/querying.html#recursive-ctes)
```
def get_categories(category=None):
    Base = Category.alias()
    level = Value(1).alias("level")
    path = Base.name.alias("path")
    base_case = (Base.select(Base.id, Base.name, Base.icon, Base.parent, level,
                             path).where(Base.parent == category).cte(
                                 "base", recursive=True))

    RTerm = Category.alias()
    rlevel = (base_case.c.level + 1).alias("level")
    rpath = base_case.c.path.concat("/").concat(RTerm.name).alias("path")
    recursive = RTerm.select(RTerm.id, RTerm.name, RTerm.icon, RTerm.parent, rlevel,
                             rpath).join(base_case,
                                         on=(RTerm.parent == base_case.c.id))

    cte = base_case.union_all(recursive)

    query = cte.select_from(cte.c.name, cte.c.icon, cte.c.level, cte.c.path,
                            cte.c.id, cte.c.parent_id).order_by(cte.c.level)

    return query
```

Category class:
```
class Category(Model):
    name = CharField()
    icon = CharField(null=True)
    parent = ForeignKeyField("self", null=True, backref="children")
```

And last but not least, error message:
```
Traceback (most recent call last):
  File "<string>", line 28, in <module>
  File "C:\Users\jp\.nuke\QUAM\Libs\peewee\peewee.py", line 4378, in next
    self.cursor_wrapper.iterate()
  File "C:\Users\jp\.nuke\QUAM\Libs\peewee\peewee.py", line 4297, in iterate
    result = self.process_row(row)
  File "C:\Users\jp\.nuke\QUAM\Libs\peewee\peewee.py", line 4363, in process_row
    return self.constructor(**row_dict)
  File "C:\Users\jp.nuke\QUAM\Libs\peewee\peewee.py", line 6736, in __call__
    return self.model(**kwargs)
  File "C:\Users\jp\.nuke\QUAM\Libs\peewee\peewee.py", line 6260, in __init__
    setattr(self, k, kwargs[k])
AttributeError: can't set attribute
```

Greetings, peewee is AWESOME :D

The code you've highlighted is setting attributes dynamically onto an object, and this is a fairly fundamental thing to Peewee's operation. Possibly this is being prevented because these attributes correspond with the field instances declared on the model class? At any rate, this seems to be something you should bring up with the author of the python implementation you're attempting to use.
Issue: Connections fail with: peewee.OperationalError: (1524, "Plugin 'unix_socket' is not loaded")
Author: scjoiner
State: closed
Created: 2021-04-05 14:22:46

In Peewee 3.14.4 connections are failing with the error message "peewee.OperationalError: (1524, "Plugin 'unix_socket' is not loaded")" on the db.connect() command. 

OS is Ubuntu 18.04 running Python 3.6 in a virtual environment for a Flask application with uwsgi. 

This application was previously working and this is a new issue.

Sounds like an issue in your db driver. That error is being propagated up from whatever driver you're using. Presumably you upgraded something else in addition to just peewee. You can diff the last working version with 3.14.4 and see if anything jumps out at you, but it seems unlikely to me.
Issue: model_to_dict exclude
Author: bronte2k7
State: closed
Created: 2021-04-05 09:32:04

Hi

```python
class Account(Model):
	email = CharField(max_length=255, unique=True, index=True)
	password = PasswordField(index=True)

account = Account.get_or_none(Accountl.id == 1)
account = model_to_dict(account, exclude=['password'])
```

Password is not excluded


http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#model_to_dict

>exclude – A list (or set) of field instances which should be excluded from the result dictionary.

You need to pass the field instance.

I do not understand something. Please show me an example. I have already tried it in different ways))) 

Going forward, please post questions to stackoverflow.

>example

```python
account = model_to_dict(account, exclude=[Account.password])
```

thank you!
Issue: contrib a DillField ?
Author: aiqc
State: closed
Created: 2021-04-03 22:20:13

I've been using the `PickleField` a lot, but ran into Pickle's limitations. These are overcome by `dill`, which was written by the `pathos` people.
https://dill.readthedocs.io/en/latest/#major-features

At first, I thought I would just save my dill-serialized objects as bytes in the `BlobField()`, but then then I realized, "bah, every time I want to read this I'll need to call a method that deserializes it; lame." This was fine for internal fields, but not for user-facing fields.

```python
import io
import dill as dill

def dill_serialize(objekt:object):
    blob = io.BytesIO()
    dill.dump(objekt, blob) #regular pickle is `dumps`
    blob = blob.getvalue()
    return blob


def dill_deserialize(blob:bytes):
    objekt = io.BytesIO(blob)
    objekt = dill.load(objekt) #regular pickle is `loads`
    return objekt

# in action on a notorious nested function
def funk():
    def funkier():
        return 'z'
    return funkier()

blob = dill_serialize(funk)
unblob = dill_deserialize(blob)

unblob()
# 'z'

```

Are you open to the idea of a DillField?

`playhouse.fields`
https://github.com/coleifer/peewee/blob/a8938588c00ae7ef94f453b8873820419474b4ce/playhouse/fields.py

`tests.extra_fields`
https://github.com/coleifer/peewee/blob/a8938588c00ae7ef94f453b8873820419474b4ce/playhouse/fields.py


You're welcome to package this up as your own extension module, but I try to stay within the stdlib where possible.
Issue: `Model.first()` doc reference broken
Author: Helveg
State: closed
Created: 2021-04-03 07:20:11

In the docs under [Selecting a single record](https://github.com/coleifer/peewee/blob/master/docs/peewee/querying.rst#selecting-a-single-record) there is a broken link to `Model.first`. Using it yields an attribute error that `first` doesn't exist on my model. I fixed this by using `Model.select().first()` and most likely the docs also need to reference the `first` method of the `ModelSelect` class instead?
Issue: Add ModelSelect.get_or_none()
Author: ilyakamens
State: closed
Created: 2021-04-02 00:50:29

Mr. Leifer,

First, thanks for creating and maintaining such a great library. We've used it at Propel for many years now.

This PR adds `get_or_none()` to `ModelSelect` in an effort to make the querying API a little more consistent. `Foo.get_or_none()` is very convenient, and we'd like the same sort of convenience for things like `Foo.select().join(Bar).where(Foo.baz == baz).get_or_none()`.

Thanks for the patch, looks good and merged.
Issue: execute count
Author: leshphonc
State: closed
Created: 2021-04-01 09:16:30

![image](https://user-images.githubusercontent.com/24892968/113272208-f76e7000-930d-11eb-94ec-6dc94a9f4972.png)


是我用法的问题吗，请帮帮我

我本地运行是没问题的，放到服务器上就获取不到count了

Do not call `.execute()` before count().

Execute gives you a cursor iterator, not a composable query object.

```python

# This gives us a query object.
query = User.select().where(User.status == 0)

# Executes a SELECT COUNT(*)
count = query.count()

# Not necessary to explicitly call .execute() - as iterating over
# a select query implicitly executes it.
for user in query:
   ...
```
Issue: Cannot nest database as context manager in transactions
Author: arjd
State: closed
Created: 2021-03-23 22:22:00

It seems to me that this should be possible:

```
with db.atomic():
    with db:
        inst = Model.create()
```

But this fails with `OperationalError: Attempting to close database while transaction is open.`.

This is pretty similar to https://github.com/coleifer/peewee/issues/1918 

What is the intent of such a construction? Using the database as context manager implies you wish to close the database when it exits. Peewee is correctly telling you it cannot close the database because you have explicitly requested a transaction.

Not a bug, in my opinion.
Issue: Not .select()'ed foreign key field raises DoesNotExist with lazy_load=False
Author: AndBondStyle
State: closed
Created: 2021-03-20 14:41:41

I have a model with a foreign key field like this:
```python3
related_id = ForeignKeyField(OtherModel, lazy_load=False)
```
If you don't include this field in a select query and then try to access it, you get `DoesNotExist` error:
```python3
x = Model.select(Model.id).get()  # related_id field not included
print(x.related_id)  # raises DoesNotExist
```
... which is not very expected, considering `lazy_load=False`.  

The root of exception is in the `ForeignKeyAccessor.get_rel_instance` method, and I believe it should be changed like this:
```diff
def get_rel_instance(self, instance):
    value = instance.__data__.get(self.name)
    if value is not None or self.name in instance.__rel__:
        if self.name not in instance.__rel__ and self.field.lazy_load:
            obj = self.rel_model.get(self.field.rel_field == value)
            instance.__rel__[self.name] = obj
        return instance.__rel__.get(self.name, value)
-    elif not self.field.null:
+    elif not self.field.null and self.field.lazy_load:
        raise self.rel_model.DoesNotExist
    return value
```



Looks like a regression related to #2328

Actually scratch that - this is not a regression. There doesn't seem to be a reliable way of differentiating between a model instance you have explicitly chosen *not* to select the foreign-key value and a model instance that is missing a non-null foreign-key. Lazy-load really deals with converting a foreign-key value (e.g. integer id) into a full object.

That being said, I agree that the error is misplaced and this should just return `None` if lazy-load has been disabled. I've made the patch as you suggested.