akumoth/peewee-issues huggingface dataset:
Issue: A replacement for playhouse.shortcuts.RetryOperationalError?
Author: cubranic
State: closed
Created: 2018-02-05 23:47:14

Commit 54a672547f549eb381cd9578a2b19310bcf5ef1d removed the class `playhouse.shortcuts.RetryOperationalError`. Is there a new recommended way to automatically reconnect and retry on failed queries?

The code should work with a few modifications...the signature for `execute_sql()` is slightly different. `exception_wrapper` is now `__exception_wrapper__`. `get_cursor()` is just `cursor()`...

You'll have to experiment, but look at the way `execute_sql()` is implemented and I'm sure you can sort it out.

Can you tell us why RetryOperationalError is removed? @coleifer 

Removed as I have no intention of continuing to maintain it...also, it's my opinion that it's a gross hack.

This should work if you have legacy code.

```python
from peewee import MySQLDatabase
from peewee import OperationalError


class RetryOperationalError(object):

    def execute_sql(self, sql, params=None, commit=True):
        try:
            cursor = super(RetryOperationalError, self).execute_sql(
                sql, params, commit)
        except OperationalError:
            if not self.is_closed():
                self.close()
            with __exception_wrapper__:
                cursor = self.cursor()
                cursor.execute(sql, params or ())
                if commit and not self.in_transaction():
                    self.commit()
        return cursor


class MyRetryDB(RetryOperationalError, MySQLDatabase):
    pass

```

Small note:

``__exception_wrapper__`` should not have parentheses afterwards -- you don't need to call it. I went ahead and edited your code snippet to make the change.

The work around seems to have broken. Was `self.__exception_wrapper__` renamed to something else, or removed entirely?


Sorry, ``__exception_wrapper__`` is just at module scope.

So,

```python
import peewee

with peewee.__exception_wrapper__:
    ...
```
Issue: pwiz doesn't properly generate models for postgresql array columns
Author: james-lawrence
State: closed
Created: 2018-02-05 17:52:40

given a table with an text array column pwiz generates `UnknownField #ARRAY` instead of `ArrayField(TextField)`.

This is a known limitation of the introspection tools. I'm not sure how to go from the OID returned by the cursor description to a suitable definition of an ArrayField object.

then leave this issue open. =)

side note: each array type has its own OID, you just need to map them:
here is an example list of OIDs pulled from https://github.com/jackc/pgx/blob/master/pgtype/pgtype.go,
there is a query you can run to pull the list from the DB as well.
```
	BoolOID             = 16
	ByteaOID            = 17
	CharOID             = 18
	NameOID             = 19
	Int8OID             = 20
	Int2OID             = 21
	Int4OID             = 23
	TextOID             = 25
	OIDOID              = 26
	TIDOID              = 27
	XIDOID              = 28
	CIDOID              = 29
	JSONOID             = 114
	CIDROID             = 650
	CIDRArrayOID        = 651
	Float4OID           = 700
	Float8OID           = 701
	UnknownOID          = 705
	InetOID             = 869
	BoolArrayOID        = 1000
	Int2ArrayOID        = 1005
	Int4ArrayOID        = 1007
	TextArrayOID        = 1009
	ByteaArrayOID       = 1001
	BPCharArrayOID      = 1014
        VarcharArrayOID = 1015
```

also if the problem is figuring out the OID of the table columns there is a query for that as well.

:+1: 

Support for arrayfield added in: c6b17b59cb491470b975c83ab23fd6e28311a4d2
Issue: Cannot nest select within insert in 3.0.11
Author: tboz203
State: closed
Created: 2018-02-05 17:13:51

Using python 3.5.1 (with pysqlite 2.6.0, sqlite 3.8.11) and peewee 3.0.11

Using an `INSERT` statement like the one below generates sql where the nested `SELECT` expression is not enclosed in parentheses:

```python
query_id = Table.select(Table.id).where(Table.value == "whatever")
Table.insert(id=query_id, value="whatever (improved!)").on_conflict_replace()
```

```sql
INSERT OR REPLACE INTO "Table" ("id", "value") 
    VALUES (SELECT "Table"."id" FROM "Table" AS "Table" WHERE ("Table"."value" = 'whatever'), 'whatever (improved!)');
```

The exception this throws is `peewee.OperationalError: near "SELECT": syntax error`

Attaching a test script, with (lightly scrubbed) output.
[peewee_insert_select_test.py.txt](https://github.com/coleifer/peewee/files/1696070/peewee_insert_select_test.py.txt)
[output.txt](https://github.com/coleifer/peewee/files/1696063/output.txt)



Committed 8131847467a7be0f44e22f5ceb64470bb8586f93, which fixes the issue. Thank you for reporting, I've also pushed version 3.0.12 to PyPI, which includes the fix.

Fantastic! many thanks.
Issue: Pylint: "No value for argument 'database' in method call" on count() call
Author: mikekeda
State: closed
Created: 2018-02-03 19:26:05

In my code I have `User.select().count()`, after upgrading Peewee from 2.10.2 to 3.0.10 I'm getting pylint error:

> [E1120(no-value-for-parameter), users_page] No value for argument 'database' in method call

Can we add default value None for database in count method?
```
    @database_required
    def count(self, database=None, clear_limit=False):
```
https://github.com/coleifer/peewee/blob/master/peewee.py#L1695


P.S. As I see a lot of methods now have `database` argument with `@database_required` decorator, so there will be pylint errors too.

P.P.S `count()` can be called as `User.select().count(None)` without Pylint error, this issue can be closed. 

This is python, a dynamic language. The linter is, quite simply, wrong. Not fixing because there's no bug here. Just silly tools.
Issue: Fix not_in with empty list
Author: aconz2
State: closed
Created: 2018-02-02 22:08:17

Using `not_in` with an empty list produces malformed SQL

```python
import peewee
from peewee import *

class MyBase(Model):

    class Meta:
        database = SqliteDatabase(':memory:')

class Foo(MyBase):
    name = CharField()

print(peewee.__version__)
print(Foo.select().where(Foo.name.not_in([])).sql())
```

```
2.10.2
('SELECT "t1"."id", "t1"."name" FROM "foo" AS t1 WHERE ("t1"."name" NOT IN ())', [])
```
```
3.0.10
('SELECT "t1"."id", "t1"."name" FROM "foo" AS "t1" WHERE ("t1"."name" NOT IN )', [])
```

I think (?) that is the right fix. And added a new test to cover it

Postgresql treats `IN ()` (and incidentally `NOT IN ()`) as invalid SQL expressions. Because the semantics of `IN ()` are equivalent to false, Peewee replaces `IN ()` with `1 = 0`. I suppose it should do similar rewriting for `NOT IN ()`, converting it to `1 = 1` since it is always true.

That being said, it seems like the proposed change may be correct.

Replaced your patch with bc61d3fa515c462cb86714e00ffdf728121a90f7 -- which handles the IN/NOT IN special-case correctly. I **did**, however, choose to keep your changes to `NodeList.__sql__` as I think they're semantically correct.
Issue: Database.get_cursor missed in 3.X version
Author: vikt0rs
State: closed
Created: 2018-02-02 16:57:00

Class Database has a "get_cursor" method in peewee 2.X versions, see [0] for example. But for some reason, it missed in 3.X versions. Documentation and changelog say nothing about this change.

Please, suggest. Thanks!

[0] - https://github.com/coleifer/peewee/blob/2.10.2/peewee.py#L3773

It's `Database.cursor()` now. Updated docs to reflect this.
Issue: Pass kwargs through Function decorator. Fixes #1462
Author: Karel-van-de-Plassche
State: closed
Created: 2018-02-02 13:47:24

This enables the user to do for example

``` python
fn.ARRAY_AGG(Some.thing, coerce=False)
```

I've restricted the kwargs to just "coerce" and merged a similar patch. Thank you for pointing this out: 51cb6bdc25f62916fbc1f9bc471f1a1bccc1a7da
Issue: Equality check on postgreSQL ArrayField clarification
Author: Karel-van-de-Plassche
State: closed
Created: 2018-02-02 11:57:24

Continuation of https://github.com/coleifer/peewee/issues/1311
> When doing a SELECT, peewee doesn't know if the list you are passing should be unpacked (as it would for an IN query, for example). To work around, when passing lists in this situation you should wrap it in Param:

Isn't it possible for peewee to detect a PostgreSQL ArrayField is involved, and thus the operands should not be unpacked? This would give a more consistent user experience. Alternatively, maybe it's good to document this workaround somewhere (or include it in the unit tests, as that is a place people will look for examples)

> Isn't it possible for peewee to detect a PostgreSQL ArrayField is involved, and thus the operands should not be unpacked?

What behavior are you seeing in relation to ArrayField?

I get:
``` python
In [1]: Network.target_names.__class__
Out[1]: playhouse.postgres_ext.ArrayField

In [2]: Network.select().where(Network.target_names == ['efeITG_GB']).sql()
Out[2]:
('SELECT "t1"."id", "t1"."filter_id", "t1"."train_script_id", "t1"."feature_prescale_bias", "t1"."feature_prescale_factor", "t1"."target_prescale_bias", "t1"."target_prescale_factor", "t1"."feature_names", "t1"."feature_min", "t1"."feature_max", "t1"."target_names", "t1"."target_min", "t1"."target_max", "t1"."timestamp" FROM "develop"."network" AS "t1" WHERE ("t1"."target_names" = (%s))',
 [['e', 'f', 'e', 'I', 'T', 'G', '_', 'G', 'B']])
```
Where I would expect the SQL as generated by the mentioned workaround:
``` python
In [4]: Network.select().where(Network.target_names == AsIs(['efeITG_GB'])).sql()
Out[4]:
('SELECT "t1"."id", "t1"."filter_id", "t1"."train_script_id", "t1"."feature_prescale_bias", "t1"."feature_prescale_factor", "t1"."target_prescale_bias", "t1"."target_prescale_factor", "t1"."feature_names", "t1"."feature_min", "t1"."feature_max", "t1"."target_names", "t1"."target_min", "t1"."target_max", "t1"."timestamp" FROM "develop"."network" AS "t1" WHERE ("t1"."target_names" = %s)',
 [['efeITG_GB']])
```
I'm not sure if this should be natively handled by peewee, but I think equality check with an ArrayField is a common thing to do.

Postgresql array equality is fixed in 4a14174 (and inequality/gt/lt/ge/le).

More commonly, though, folks use the containment operators when working with arrays. So this hasn't really come up before.

Thanks! I did not notice this in the howto's/guides, but to be honest it has been a while since I read them thoroughly.
Issue: count() on ModelCompoundSelectQuery raises error
Author: Karel-van-de-Plassche
State: closed
Created: 2018-02-02 11:45:22

I'm rewriting code from 2.10.1 to 3.0.9. One used to be able to do a .count() on a combined query, but now it raises an `AttributeError`. For example:
``` python
(Network.select().where(Network.id == 301) |
 Network.select().where(Network.id == 300)).count()
```
``` python
~/working/peewee/peewee.py in inner(self, database, *args, **kwargs)
   1440             raise Exception('Query must be bound to a database in order '
   1441                             'to call "%s".' % method.__name__)
-> 1442         return method(self, database, *args, **kwargs)
   1443     return inner
   1444 

~/working/peewee/peewee.py in count(self, database, clear_limit)
   1695             clone._limit = clone._offset = None
   1696         try:
-> 1697             if clone._having is None and clone._windows is None and \
   1698                clone._distinct is None and clone._simple_distinct is not True:
   1699                 clone = clone.select(SQL('1'))

AttributeError: 'ModelCompoundSelectQuery' object has no attribute '_having'
```
A working workaround is to wrap the offending boolean check in a `try: except:` clause, but I suppose the `CompoundSelectQuery` class should just initialize these attributes, as I feel it should call the initializer from `Select`.

PS. This works without issues:
``` python
len(Network.select().where(Network.id == 301) |
 Network.select().where(Network.id == 300))
```

Ahhh, interesting. Will get this fixed.

Fixed and added a test-case.
Issue: Orphaned code in peewee-3.0
Author: conqp
State: closed
Created: 2018-02-02 08:50:12

While debugging some problems with peewee-3.0 I discovered some strange code, e.g. https://github.com/coleifer/peewee/blob/406eb5865d72f1c74f48abd67bb8f4c9ccc1391a/peewee.py#L4095 
The `_get_pk_value()` method was part of peewee < 3.0, but is no longer defined on peewee-3.0's `Model`. Heck, it isn't even defined anywhere in `peewee.py`.
Is this orphaned code or am I missing something?

Yipes, I was scared for a sec that I had a glaring gap in the tests. Looking closer I think the `ForeignKeyAccessor` should handle setting the value so that this code is, indeed, dead. But for now I've simply updated the method to ".get_id()", which is the correct name.
Issue: Allow updating existing model via dict_to_model
Author: tuukkamustonen
State: closed
Created: 2018-02-02 08:32:45

Hosting a HTTP API, `PATCH` requests update existing data. Adding `instance` or similar to `dict_to_model` would allow to update the loaded model:

```python
def dict_to_model(model_class, data, ignore_unknown=False):
    instance = model_class()
```

As in:

```python
def dict_to_model(..., instance=None, ...):
    instance = instance if instance else model_class()
    ...

obj = Foo.get(id=...)
dict_to_model(Foo, payload, instance=obj)
```

I guess `model_class` would not be needed in case `instance` is passed in?

I implemented something like that some time ago:
https://github.com/HOMEINFO/peeweeplus/blob/055be28cc1ce75210772a4c097be1361fe7db6dc/peeweeplus/json.py#L172

Implemented it as a separate function. `dict_to_model` works the same as it did previously and to update an existing instance, you can use:

`update_model_from_dict(instance, data[, ignore_unknown=False])`.

Documentation has been added, see: http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#shortcuts
Issue: from playhouse.shortcuts import case
Author: ra-esmith
State: closed
Created: 2018-02-02 00:55:00

I am porting up from peewee 2 to 3 and use the case statement. 
I get an import error now ...

What is the work around for case?

Thanks,
Evan

Ah I see
The Case() helper has moved from the playhouse.shortcuts module into the main peewee module.

Not sure what that import is, but great I will find it

If you use `from peewee import *` then the Case helper is automatically imported. `Case` is the same as `case` was in 2.x, so it should be as simple as removing the old imports and switching from "case" to "Case".

 from peewee import case
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name case


ok 
will try import *, though I tend to avoid * when I can
thanks!!

It's "Case" (upper-case "c").

Re: "from peewee import *", Peewee defines `__all__` at module-scope so the things that are imported are limited to a subset of everything in the peewee module namespace. It's a matter of preference, tho, so whatever works.
Issue: cascade with drop_table
Author: ra-esmith
State: closed
Created: 2018-02-01 23:56:15

Porting up from 2 to 3 looking for cascade option

.db.database.drop_tables(ALL_MODELS, safe=True, cascade=True)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/peewee.py", line 2579, in drop_tables
    model.drop_table(**kwargs)
TypeError: drop_table() got an unexpected keyword argument 'cascade'

ahh looks like you just fixed it in 3.0.9
https://github.com/coleifer/peewee/commit/d49784c2d6e525d882aab702fe0d2cf8163e4c18

will see if i can grab that

hmm, tried 3.0.9 and even took what is in git, still get unexpected keyword.  Though looks like it should work ... 

Let me look into it some more.

Fixed.
Issue: Implementing database context in peewee 3.0
Author: conqp
State: closed
Created: 2018-02-01 12:20:16

Unfortunately I have been to hasty migrating to peewee 3.0.

I was using an [extension](https://github.com/HOMEINFO/peeweeplus/blob/27aef530f3a871593aa06b9b940057a033afa0ce/peeweeplus/database.py) of the MySQLDatabase class that handles connections well (if invoked with `closing=True`).
This hack was/is supposed to compensate for peewee < 3.0 's bad behaviour regarding connection handling.

Do you have any suggestions, how I may migrate this to the new API of peewee 3.0 (which unfortunately still has no useful docstrings documenting the respective classes, functions and parameters)?

My goal is the following:
I do not want peewee to raise `Error 2006` after a certain runtime as described [here](http://peewee.readthedocs.io/en/latest/peewee/database.html#error-2006-mysql-server-has-gone-away).
Unfortunately, `MySQLDatabase.execution_context ` does no longer exist.

Thanks for any hint.

The [API documentation](http://docs.peewee-orm.com/en/latest/peewee/api.html) is **much** more thorough than the 2.x documentation, have you taken a look there?

Similarly the [changes documentation](http://docs.peewee-orm.com/en/latest/peewee/changes.html) describes that, in place of `execution_context()`, you can:

* Simply use the database instance as a context manager, in which case the wrapped code is also wrapped in a transaction
* Use `database.connection_context()`, which has similar semantics to `execution_context()`.

Does this solve your problem?

Hi. Tanks for the hint regarding the changes page. I actually did not notice that.
`database.connection_context()` looks promising. I'll re-open if it'll turn out to not work.
Issue: update with inner join
Author: ra-esmith
State: closed
Created: 2018-01-31 01:19:26

hello,

what is the syntax for a peewee query such as the following?  Is there a good way to figure this out on my own (like an sql to query syntax generator?).
```sql
update
    customer
 set
    customer.x = customerrep.x,
    customer.y = customerrep.y
from
    customer cust
inner join
    customerrep custrep
on 
    cust.acct = custrep.acct
```

#my thought is to try

```python
def test():
    with my_app.db.database.atomic():
        Customer.update(x=CustomerRep.select(CustomerRep.x)
                        .where(CustomerRep.acct == Customer.acct)
                        .order_by(CustomerRep.id.desc())
                        .limit(1)).execute()
        Customer.update(y=CustomerRep.select(CustomerRep.y)
                        .where(CustomerRep.acct == Customer.acct)
                        .order_by(CustomerRep.id.desc())
                        .limit(1)).execute()
```


Did you happen to notice, when commenting or creating your issue that your code was not formatted? Please take the extra 10 seconds to do this when submitting issues.

So, unfortunately, Peewee does not really support updates with joins. In fact, I don't know if any of the RDBMS's that Peewee supports allow that syntax.

* https://www.postgresql.org/docs/current/static/sql-update.html
* https://www.sqlite.org/lang_update.html

Check out this example for some ideas: http://docs.peewee-orm.com/en/latest/peewee/query_examples.html#update-a-row-based-on-the-contents-of-another-row
Issue: Fix how set's are translated to SQL
Author: aconz2
State: closed
Created: 2018-01-30 23:24:19

The following code breaks from 2 to 3

```python
import peewee
from peewee import *

db = SqliteDatabase(':memory:')

class MyModel(Model):
  class Meta:
    database = db

class Foo(MyModel):
  name = CharField()

print(peewee.__version__)
print(Foo.select().where(Foo.name.in_({'foo'})).sql())
```

And the output for each
```
2.10.2
('SELECT "t1"."id", "t1"."name" FROM "foo" AS t1 WHERE ("t1"."name" IN (?))', ['foo'])

```
```
3.0.5
('SELECT "t1"."id", "t1"."name" FROM "foo" AS "t1" WHERE ("t1"."name" IN ?)', ["{'foo'}"])
```

Given how the conversion used to include `set`s ([this](https://github.com/coleifer/peewee/blob/2.10.2/peewee.py#L1852) is the relavent line I think), this would be the "fix".

If that change was intentional I understand that as well.

Fixed with 7af636d8dd81573cf02ede0bbcb6f605c8ab6118 -- which includes test. Thanks.
Issue: PasswordField & PickledField on 3?
Author: ghost
State: closed
Created: 2018-01-30 22:15:36

http://docs.peewee-orm.com/en/latest/peewee/changes.html#fields

What are the alternatives for `PasswordField` and `PickledField` on 3.x ?.
What are you using if you need `PasswordField` and `PickledField` on 3.x ?.

The Documentation dont mention alternatives.
:thinking: 


Hey, those are no longer supported. PasswordField had some legitimate security concerns raised about it, and I don't want to give the false sense of security as to what it claims to do. PickleField is quite easy to implement in your own code or in a library.

```python
import sqlite
import pickle

class PickledField(BlobField):
    def python_value(self, value):
        if isinstance(value, (bytearray, sqlite3.Binary)):
            value = bytes(value)
        return pickle.loads(value)

    def db_value(self, value):
        return sqlite3.Binary(pickle.dumps(value, 2))
```

**Thank you very much!.**

Field now lives here, in case someone still need it, or to ease migration to 3.x:
https://github.com/juancarlospaco/peewee-extra-fields#pickledfield
:smiley:


Good stuff brotha, thanks

So should we use https://github.com/juancarlospaco/peewee-extra-fields#simplepasswordfield instead?

Just in case anyone was troubleshooting PickleField
http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#PickleField

```
from playhouse.fields import PickleField
```
Issue: 3.0.2 release breaks FlaskAdmin
Author: brunson
State: closed
Created: 2018-01-30 16:00:46

When upgrading to peewee 3.0.2 from 2.10.2 the contrib.peewee module for flask_admin fails attempting to import PrimaryKeyField from peewee.


I'm not the maintainer of Flask-Admin. You can see in the [changes](http://docs.peewee-orm.com/en/latest/peewee/changes.html#fields) that PrimaryKeyField has been renamed to AutoField.

To simplify updating, I suppose I can add a shim that logs a DeprecationWarning if you try to use PrimaryKeyField, however. This will give other maintainers some warning to upgrade their code. WDYT?

Pushed 3.0.4, which adds a shim for `PrimaryKeyField`, which raises a DeprecationWarning if you attempt to use it. This should give folks time to update their codes.

Wow, that's awesome.  I created an issue for the flask_admin maintainer:  https://github.com/mrjoes/flask-admin/issues/42

Thanks for the amazing turn around!

Of course, thanks for reporting the issue. 3.0 is a major release and I want to be sure to make the transition as smooth as possible for folks upgrading. Cheers.