akumoth/peewee-issues huggingface dataset:
Issue: Default order in 3.x
Author: phryk
State: closed
Created: 2018-02-25 00:23:26

The "Changes in 3.0" documentation page states that `_meta.order_by` has been removed,
but it doesn't state anything on how to set a default ordering in 3.x and I haven't found anything on this in the code either.

Is there now a better way of doing this or was this feature just dropped without a substitute?

If the latter is true, would you consider bringing it back? I always considered `_meta.order_by` to be a good DX boon. :<

The feature was dropped due to it's perceived smelliness... If you want to achieve something similar in 3.x, you can always:

```python
class User(Model):
    username = TextField()

    @classmethod
    def ordered(cls, *args):
        return cls.select(*args).order_by(cls.username)

unordered = User.select().where(...)
ordered = User.ordered().where(...)
```

I miss this feature too, though I do agree that it's a bit smelly. Nevertheless, it can be very handy, especially when you're trying to get a quick and dirty web app put together. One notable example is in flask-admin, where you want the dropdown menus for automatically scaffolded foreign key fields to be in a sane order.

Just wanted to share my quick re-implementation of this feature for anyone else that might be looking for it:

```python
class BaseModel(ModelBase):

    class Meta:
        order_by = []  # override this value in subclasses, e.g. `order_by = ['name']`

    # Simple re-implementation of default ordering that was removed in peewee 3.x
    # Ideally, ordering should always be explicitly set in query, but this can be
    # very handy, esp. for auto-scaffolding frameworks like flask-admin
    @classmethod
    def select(cls, *fields):
        order_by = []

        for field_name in cls._meta.order_by:
            if field_name.startswith('-'):
                order_by.append(getattr(cls, field_name[1:]).desc())
            else:
                order_by.append(getattr(cls, field_name))

        return super(BaseModel, cls).select(*fields).order_by(*order_by)
```

I would think `flask-admin` doesn't provide a way to define or override the query of related objects, but *shrug*?
Issue: Undefined symbol ReverseRelationDescriptor in peewee.py
Author: rammie
State: closed
Created: 2018-02-22 15:40:33

The symbol `ReverseRelationDescriptor` does not seem to appear anywhere in peewee.py except here:

https://github.com/coleifer/peewee/blob/94aee08a1fee79622148cb1fcd3d2af81a834e04/peewee.py#L5724

Just thought I'd point it out as it seems to have been left behind in error (or I'm missing something).

Thanks for reporting, I must've missed this during the rewrite. Fixed in master now.

Looks like there are still issues with the `filter()` API...looking into it.
Issue: Field class no more supports add_to_class(class, name) method
Author: skyhound
State: closed
Created: 2018-02-22 09:30:20

In peewee 2.x it was possible to dynamically define fileds in model using this method, like it's described here:
https://stackoverflow.com/questions/22358489/dynamically-define-fields-in-a-peewee-model
However, in 3.x this method doesn't work. Is there some alternative?

You can use `Model._meta.add_field(field_name, field)`.

> You can use `Model._meta.add_field(field_name, field)`.

Not able to add field in database using this. How can I do it??

> Not able to add field in database using this. How can I do it??

Probably you need to use a schema migration? http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#schema-migrations

Is any way to get only fields of class ??
Ex - 
class Test1(BaseModel):
    name = CharField(default = 'Test')
    new_column = peewee.IntegerField(default = 0)
    new_column1 = IntegerField(default = 0)
    new_column2 = IntegerField(default = 0)
    new_column3 = IntegerField(default = 0)
    new_column4 = IntegerField(default = 0)

output expected - 
['name', 'new_column', 'new_column1', 'new_column2', 'new_column3', 'new_column4 ']
--




> > Not able to add field in database using this. How can I do it??
> 
> Probably you need to use a schema migration? http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#schema-migrations

Thanks coleifer, It worked...

@jayashreesathe http://lmgtfy.com/?q=Get+fields+on+model+Peewee+python
Issue: Documentation issues for 3.0
Author: phryk
State: closed
Created: 2018-02-21 18:01:07

I'm currently porting my project over from peewee 2 to 3 and stumbled upon a few things already.

A few renames not listed on the "Changes in 3.0" page that affected me:
* `Field.add_to_class` has apparently been reworked to `Field.bind`?
* `Model._[get|set]_pk_value` has been replaced with the dynamic `Model._pk` property
* `FieldDescriptor` has been renamed `FieldAccessor`, similar renames for all child-classes

Also, `Clause` has been dropped (or reworked into `NodeList`?) but is still referenced on the "Hacks" page.

Lastly, `Model.prepared` has been completely dropped and I don't see any substitute for it, do you have any recommendations on what to use?



The APIs you're referring to were not ever public APIs, hence the changes to them are not documented. I'll update the "Hacks" page to correct the reference to `Clause` (which is now `NodeList`).

`Model.prepared` has been removed -- depends on what you're trying to do. You can always override the `__init__()` method.

a309831f50e1a090204024f2a738c3c6f333b6fc

Adding or removing a field can now be done using `Model._meta.add_field` or `Model._meta.remove_field`.

I'm using it in a bunch of places, mostly for adding some extra properties to model instances that depend on the data from the db.

Maybe I should just make these things `@property`s…
Issue: Create DB Transaction manually (without a context manager)
Author: stanislavkozlovski
State: closed
Created: 2018-02-20 17:44:02

Hey-o. I've been trying to wrap all my pytest unit tests in a transaction and then roll them back but have failed in doing so.

The guide I've been trying to follow has been this one: http://techblog.thescore.com/2015/06/25/database-transactions-with-pytest/

The problem here is that I write my transaction creating code in a `pytest_fixture` but the context manager exits before we manage to call the `fin()` function to close the transaction
```
@pytest.fixture(autouse=True)
    def _wrap_test_in_transaction(self, request):
        with db.manual_commit():
            db.begin()
            def fin():
                db.rollback()

            request.addfinalizer(fin)
```

The ability to open a transaction without a context manager would fix this problem for me

Peewee provides such nice facilities for transaction management (and rollback) ... It's a shame you're not using them.

If you want to open a "manual" block without using the context manager, just:

```python
manual = db.manual_commit()
manual.__enter__()
```

To close:

```python
manual.__exit__(None, None, None)
```

Thank you, this should work. I feel ashamed I missed this yesterday

No worries!
Issue: Not failing silently when creating tables on attached SQLite databases
Author: maurosilber
State: closed
Created: 2018-02-18 23:25:39

Here's minimal example reproducing the issue:

```python
from playhouse.apsw_ext import *

db = APSWDatabase('main.db')
db.execute_sql('attach database "other.db" as other')

class MainTable(Model):
    field1 = IntegerField()
    
    class Meta:
        database = db
    
class AttachedTable(Model):
    field1 = IntegerField()
    
    class Meta:
        database = db
        schema = 'other'

MainTable.create_table(fail_silently=True)
AttachedTable.create_table(fail_silently=True)
```

```python
File "<ipython-input-5-002497356290>", line 22, in <module>
    AttachedTable.create_table(fail_silently=True)

File "python3.5/site-packages/peewee.py", line 4857, in create_table
    db.create_table(cls)

File "python3.5/site-packages/playhouse/sqlite_ext.py", line 948, in create_table
    return self.execute_sql(sql, params)

File "python3.5/site-packages/playhouse/apsw_ext.py", line 116, in execute_sql
    self._execute_sql(cursor, sql, params)

File "python3.5/site-packages/playhouse/apsw_ext.py", line 109, in _execute_sql
    cursor.execute(sql, params or ())

File "src/cursor.c", line 1019, in APSWCursor_execute.sqlite3_prepare

File "src/statementcache.c", line 386, in sqlite3_prepare

SQLError: SQLError: table "attachedtable" already exists
```

Never mind. It works after updating from 2.8.5 to 3.0.18.
Issue: Why does not peewee support mysql-connector-python?
Author: polikeiji
State: closed
Created: 2018-02-17 06:06:24

I think peewee support currently only MySQLdb and pymsql as drivers for MySQL. I have not compared   MySQL drivers in details, but I want to use the mysql-connector-python because I think it is the driver Oracle is maintaining officially.

I'm not familiar with mysql-connector-python, but Peewee should work with any DB-API 2.0-compatible driver. If the mysql-connector-python library is fully DB-API 2.0 compatible, you should be able to simply subclass `MySQLDatabase` and override the `_connect()` method so it uses the mysql connector.

You can refer to this document for more information: http://docs.peewee-orm.com/en/latest/peewee/database.html#adding-a-new-database-driver

You can always use PyMySQL

Peewee supports mysql-connector via `playhouse.mysql_ext.MySQLConnectorDatabase`.
Issue: delete triggers firing with on_conflict('replace')
Author: ghost
State: closed
Created: 2018-02-15 23:20:23

Hello.

I encountered this problem while inserting a new row, which violated a UNIQUE constraint, either through `Model.insert()` or `Model.insert_many()`. I didn't expect this behavior, since the [SQLite documentation](https://sqlite.org/lang_conflict.html) states that

> When the REPLACE conflict resolution strategy deletes rows in order to satisfy a constraint, delete triggers fire if and only if recursive triggers are enabled.

I doubled checked to see if `recursive_triggers` was disabled, but it doesn't even seem to be enabled by default.  Explicitly setting `recursive_triggers` by saying `db.pragma('recursive_triggers', 0, permanent=True)` also seems to have no effect. Is my understanding of SQLite wrong or would this be a bug on peewee's side? [Here](https://gist.github.com/Jovito/b2ad9a448a3d226d04c2d5b3c54d64b1) is an example.

I think you might be confusing triggers with the `ON DELETE` behavior, which I don't believe falls under the same rules as triggers (recursive or otherwise). To avoid the issue, it should be sufficient to remove either the "on delete cascade" bit, or to disable the foreign_keys pragma.

I was able to replicate the behavior you've described using just the sqlite shell, so this definitely isn't a peewee issue. Whether it's a bug in sqlite is probably worth bringing up on the mailing list.

Would there be a way to replace rows without deleting them first or is it necessary to disable `ON DELETE`/the `foreign_keys` pragma to avoid this problem?

I'm not sure, that's a good question for the SQLite mailing list. It does seem like the combination of:

1. Pragma foreign_keys = on
2. On delete cascade
3. Insert or Replace

Can lead to rows in the related table being deleted.

An alternative, if you don't want to change the first 2, would be to execute a normal INSERT and then, if that fails with an IntegrityError, to do an update instead.
Issue: Generalize Autofield test between sqlite and postgresql
Author: Karel-van-de-Plassche
State: closed
Created: 2018-02-15 15:43:35

In sqlite the `AutoField` used for `PrimaryKeys` is 'INTEGER', while in postgresql its 'SERIAL'. This breaks test `test_db_table` of `TestModelDDL` in `tests/schema.py`. Either this test, or the underlying AutoField has to be generalized.

Yes, it was a bad commit, the test is fixed in master now.
Issue: Fixed testcases broken by bugfix 9edf260
Author: Karel-van-de-Plassche
State: closed
Created: 2018-02-15 15:34:31

I think this test has to be changed, as primary keys are an AutoField, and Postgress' AutoField is SERIAL. This expected behaviour somehow only surfaces after 9edf260.

For sqlite the autofield IS an INTEGER, so this solves nothing. Opened issue #1499
Issue: peewee.OperationalError: unable to open database file 
Author: kitten77
State: closed
Created: 2018-02-15 14:11:41

Dont know if this is a bug or not but reporting it anyway
this happens when i try to create a db without having the subdir where the file is placed
 ```
 File "/usr/home/user/project/todo.py", line 12, in __init__
    DB = todoDB()
  File "/usr/home/admin/ragnhild/models/todoDB.py", line 35, in __init__
    with db, dbfile:
  File "/usr/local/lib/python3.6/site-packages/peewee.py", line 2405, in __enter__
    self.connect()
  File "/usr/local/lib/python3.6/site-packages/peewee.py", line 2435, in connect
    self._initialize_connection(self._state.conn)
  File "/usr/local/lib/python3.6/site-packages/peewee.py", line 2287, in __exit__
    reraise(new_type, new_type(*exc_args), traceback)
  File "/usr/local/lib/python3.6/site-packages/peewee.py", line 170, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/peewee.py", line 2434, in connect
    self._state.set_connection(self._connect())
  File "/usr/local/lib/python3.6/site-packages/peewee.py", line 2654, in _connect
    **self.connect_params)
peewee.OperationalError: unable to open database file
```
i made a quick and dirty workaround:
```
path = '/home/user/data/'
File = 'todo.db'
if os.path.isdir(path):
    pass
else:
    os.makedirs(path)
dbFile = path + File
db = SqliteDatabase(dbFile)
```

I've edited your comment to include code formatting...please take the time to format your issues. It takes two seconds.

Wontfix, peewee won't magically create directories for you.

Thank you for fixing my post I will put more attention into the formatting in the future 😊
Issue: Create schema when 'creating all' if it doesn't exist
Author: Karel-van-de-Plassche
State: closed
Created: 2018-02-15 13:00:36

As it says in the table. Not sure if we want to use the safe version of this method in `_create_table` itself, which is also possible. Also, I was not sure where to put Unit Tests for this. If you give me a location, I can write them if you want.

PS. One test fails for PostgreSQL, but it also fails without my pull request.

``` python
======================================================================
FAIL: test_db_table (tests.schema.TestModelDDL)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/karel/working/peewee/tests/schema.py", line 126, in test_db_table
    'CREATE TABLE "A_tbl" ("id" INTEGER NOT NULL PRIMARY KEY)'])
  File "/home/karel/working/peewee/tests/schema.py", line 83, in assertCreateTable
    self.assertEqual([sql] + indexes, expected)
AssertionError: Lists differ: ['CREATE TABLE "A_tbl" ("id" SERIAL NOT NULL PRIMARY KEY)'] != ['CREATE TABLE "A_tbl" ("id" INTEGER NOT NULL PRIMARY KEY)']

First differing element 0:
'CREATE TABLE "A_tbl" ("id" SERIAL NOT NULL PRIMARY KEY)'
'CREATE TABLE "A_tbl" ("id" INTEGER NOT NULL PRIMARY KEY)'

- ['CREATE TABLE "A_tbl" ("id" SERIAL NOT NULL PRIMARY KEY)']
?                              ^  ---

+ ['CREATE TABLE "A_tbl" ("id" INTEGER NOT NULL PRIMARY KEY)']
?                              ^^^^^


----------------------------------------------------------------------
```

I am going to pass on this, but thanks for the patch.
Issue: insert_many throws ValueError with list of collections.Mapping instances
Author: ghost
State: closed
Created: 2018-02-14 21:52:31

Hello. I've been using peewee in my latest project and I'm enjoying it quite a bit. Recently, I came across a problem when trying to pass instances of collections.Mapping to `insert_many()`, namely, it throws an ValueError exception. Here's an [example](https://gist.github.com/Jovito/2adbc33d7d60ddc6b6ebadccb750e8d6). From what I gathered, it seems that the offending line is: `is_dict = isinstance(row, dict)` in `insert_many()`. Changing it to `is_mapping = isinstance(row, (Mapping, MutableMapping))` solves the problem and still works if `row` is a regular dictionary. 

Fix pushed. It's not necessary to check both Mapping and MutableMapping, since MutableMappings are Mappings.
Issue: SQLITE_DATE_TRUNC_MAPPING undefined variable? 
Author: oz123
State: closed
Created: 2018-02-14 09:39:52

When using vim I encountered this:

```
F821 undefined name 'SQLITE_DATE_TRUNC_MAPPING'.
```
```
$ git checkout 2.9.2
$ grep -n SQLITE_DATE_TRUNC_MAPPING *
grep: docs: Is a directory
grep: examples: Is a directory
peewee.py:287:SQLITE_DATE_TRUNC_MAPPING = {
peewee.py:294:MYSQL_DATE_TRUNC_MAPPING = SQLITE_DATE_TRUNC_MAPPING.copy()
peewee.py:299:    assert lookup_type in SQLITE_DATE_TRUNC_MAPPING
peewee.py:303:    return dt.strftime(SQLITE_DATE_TRUNC_MAPPING[lookup_type])
peewee.py:4056:        return fn.strftime(SQLITE_DATE_TRUNC_MAPPING[date_part], date_field)
 $ grep -nr SQLITE_DATE_TRUNC_MAPPING *
peewee.py:287:SQLITE_DATE_TRUNC_MAPPING = {
peewee.py:294:MYSQL_DATE_TRUNC_MAPPING = SQLITE_DATE_TRUNC_MAPPING.copy()
peewee.py:299:    assert lookup_type in SQLITE_DATE_TRUNC_MAPPING
peewee.py:303:    return dt.strftime(SQLITE_DATE_TRUNC_MAPPING[lookup_type])
peewee.py:4056:        return fn.strftime(SQLITE_DATE_TRUNC_MAPPING[date_part], date_field)
playhouse/_speedups.pyx:51:cdef dict SQLITE_DATE_TRUNC_MAPPING = {
playhouse/_sqlite_ext.pyx:15:cdef dict SQLITE_DATE_TRUNC_MAPPING = {
playhouse/_sqlite_ext.pyx:29:    if lookup not in SQLITE_DATE_TRUNC_MAPPING:
playhouse/_sqlite_ext.pyx:57:        return result[0].strftime(SQLITE_DATE_TRUNC_MAPPING[result[1]])
```


So in version 2.9.2 it is both in peewee and playhouse ... In version 3.0.17 it is only defined in the playhouse module:
```
$ grep -nr SQLITE_DATE_TRUNC_MAPPING *
peewee.py:2888:        return fn.strftime(SQLITE_DATE_TRUNC_MAPPING[date_part], date_field)
playhouse/_sqlite_ext.pyx:641:cdef dict SQLITE_DATE_TRUNC_MAPPING = {
playhouse/_sqlite_ext.pyx:655:    if lookup not in SQLITE_DATE_TRUNC_MAPPING:
```

If I have some projects which only include `peewee.py` via a local copy, is this going to cause a problem for me ?

Made a quick fix, the method was out-of-date. This commit fixes the issue: 0416786d56f770c6cec3b9c7493fae7ffd6a75ec

Pushed a new version, 3.0.18, which includes the fix.

@coleifer thank you for the quick response!

No problems, you should have no issues vendoring the peewee.py module, as that's one of the goals for the project -- to make it easy to drop in where you need it.
Issue: Broken Docs
Author: ghost
State: closed
Created: 2018-02-14 08:03:26

Broken Docs, Easy to Fix.

![broken-doc0](https://user-images.githubusercontent.com/1189414/36193338-47f6fbea-1144-11e8-8653-4b25a930dc54.png)


http://docs.peewee-orm.com/en/latest/peewee/api.html#ModelIndex
:cat: 


Cheers, had those lines indented by an extra 2 spaces on accident. Should be fixed.
Issue: foreign key constraint incorrectly formed
Author: MNCODE
State: closed
Created: 2018-02-13 12:40:13

When I want to create a table I get the error foreign_key

> ERROR	session[5924]: (1005, 'Can\'t create table `example_database`.`example_table` (errno: 150 "Foreign key constraint is incorrectly formed")')

example_table is specified as:
```python
class Example_Table(BaseModel):
    id = PrimaryKeyField()
    example_table2 = ForeignKeyField(Modul)
    class Meta:
        db_table = 'Example_Table'
```
after i defined BaseModel:
```python
class BaseModel(Model):
    class Meta:
        database = database
```
and database is my MySQLDatabase Object.
The Problem is, why doesnt the foreign key constraint work and why are the tables all saved in Lowercase, but I defined them in Uppercase

If I run the program again, It creates the tables but gives me a duplicate key_name error 

Version: peewee==3.0.17



Solved by replacing db_table with table_name 
db_table is from an older version (2.x.x). Probably it also caused the foreignKeyConstraint Error
Issue: the mod op has problem in string formatting
Author: Twwy
State: closed
Created: 2018-02-13 06:36:19

I use the mod like docs.
```
from peewee import *
from peewee import Expression # the building block for expressions

def mod(lhs, rhs):
    return Expression(lhs, '%', rhs)
```
but this code has error
```
TypeError: not all arguments converted during string formatting
```
when I use ```%% ``` replace the  ```%```, also get error
```
  File "/home/test/..//package/peewee.py", line 1748, in parse_node
    sql, params, unknown = self._parse(node, alias_map, conv)
  File "/home/test/..//package/peewee.py", line 1717, in _parse
    sql, params = self._parse_map[node_type](node, alias_map, conv)
  File "/home/test/..//package/peewee.py", line 1620, in _parse_expression
    sql = template % (lhs, self.get_op(node.op), rhs)
  File "/home/test/..//package/peewee.py", line 1606, in get_op
    return self._op_map[q]
KeyError: '%%
```

Final I get the code passed, but I need to change something.

```
--- a/test/package/peewee.py
+++ b/test/package/peewee.py
@@ -315,7 +315,7 @@ OP = attrdict(
     BIN_AND='&',
     BIN_OR='|',
     XOR='^',
-    MOD='%',
+    MOD='%%',
     EQ='=',
     LT='<',
     LTE='<=',
@@ -1557,7 +1557,7 @@ class QueryCompiler(object):
         OP.XOR: '#',
         OP.AND: 'AND',
         OP.OR: 'OR',
-        OP.MOD: '%',
+        OP.MOD: '%%',
         OP.REGEXP: 'REGEXP',
         OP.CONCAT: '||',
     }

--- a/test/library/peewee_ext.py
+++ b/test/library/peewee_ext.py
@@ -48,3 +48,6 @@ def result_to_dict(obj):
         ele[key] = value
     return ele

+def mod(lhs, rhs):
+    return peewee.Expression(lhs, '%%', rhs)
```

It appears that you're using Peewee 2.x?

I've added a test-case in ba8b72a which seems to me to indicate that the code is correct on 3.x
Issue: DateTimeTZField returns datetimes in other timezone
Author: dswah
State: closed
Created: 2018-02-13 02:10:43

Hi there.
Thank you for the very useful library. I've used it a ton. However, ive recently run into something annoying with DateTimeTZField.

when i select records with a DateTimeTZField, i always see this field in timezone A (local tz of db), even though the datetimes were originally in timezone B.

why is this? how can i avoid this?

From: https://www.postgresql.org/docs/current/static/datatype-datetime.html

> For timestamp with time zone, the internally stored value is always in UTC (Universal Coordinated Time, traditionally known as Greenwich Mean Time, GMT). An input value that has an explicit time zone specified is converted to UTC using the appropriate offset for that time zone. If no time zone is stated in the input string, then it is assumed to be in the time zone indicated by the system's TimeZone parameter, and is converted to UTC using the offset for the timezone zone.
>
> When a timestamp with time zone value is output, **it is always converted from UTC to the current timezone zone, and displayed as local time in that zone. To see the time in another time zone, either change timezone or use the AT TIME ZONE construct (see Section 9.9.3).**

Thank you for the link to the documentation.
Is there a way to tell peewee to output a DateTimeTZField in a desired tz, as suggested in [1]?
This would be very useful.

Dani

[1](https://www.postgresql.org/docs/current/static/runtime-config-client.html#GUC-TIMEZONE)

If you bothered to read the linked documentation, you would have found:

> The TimeZone configuration parameter can be set in the file postgresql.conf, or in any of the other standard ways described in Chapter 19. There are also some special ways to set it:
>
> The SQL command `SET TIME ZONE` sets the time zone for the session. This is an alternative spelling of `SET TIMEZONE TO` with a more SQL-spec-compatible syntax.
>
> The `PGTZ` environment variable is used by libpq clients to send a `SET TIME ZONE` command to the server upon connection.

So, you can run: `db.execute_sql('SET TIME ZONE ...')`.