akumoth/peewee-issues huggingface dataset:
Issue: DateTimeField fetched with scalar() has different type in Postgres vs. SQLite
Author: jonklein
State: closed
Created: 2016-05-11 18:58:15

Fetching a DateTimeField with scalar() yields a datetime for Postgres, and a float for SQLite.  While SQLite does not have a distinct storage type for dates/times, I'd still expect the same type to be returned from Peewee.

With Postgres

```
>>> Profiles.select(fn.Max(Profiles.updated_at)).scalar()
datetime.datetime(2016, 4, 29, 16, 52, 33, 445791)
```

With SQLite

```
>>> Profiles.select(fn.Max(Profiles.updated_at)).scalar()
1462992876.32
```


Whoa...what version of SQLite is your Python driver using? You can run `from peewee import sqlite3; print sqlite3.sqlite_version_info` I believe. Are the sqlite values being inserted as timestamps? You can open a sqlite shell and check. 


As an example, here's what I get when I run a similar query against my blog's database:

``` python
In [1]: Entry.created_date
Out[1]: <peewee.DateTimeField at 0x7fc87b4ac7d0>

In [2]: Entry.select(fn.MAX(Entry.created_date)).scalar()
Out[2]: u'2016-03-22 23:16:57.058839'

In [3]: from peewee import sqlite3; print sqlite3.sqlite_version_info
(3, 11, 0)
```

Are you using a trigger to set the `updated_at` value, or specifying a default value generated by the database?


Looks like it's 3.9.2.  I was inserting is as a `time` in my first example, but updating to insert a `datetime.datetime` I receive a unicode string back, also not what I'm getting from Postgres (`datetime.datetime`).

Here are how the columns are defined in the Postgres database:

```
    created_at timestamp without time zone NOT NULL,
    updated_at timestamp without time zone NOT NULL,
```

To give some context here, I'm using SQLite just for running the test suite, and Postgres for actual app usage.


Welllll, I think you answered your own question. It's called a `DateTimeField` because it should be given `datetime` objects.

I meant to also show the use of the `convert` parameter to `scalar()` which I had sadly forgotten to document. At any rate, [I've added documentation](http://docs.peewee-orm.com/en/latest/peewee/api.html#Query.scalar), and this is the behavior:

``` python
In [1]: Entry.select(fn.MAX(Entry.created_date)).scalar(convert=True)
Out[1]: datetime.datetime(2016, 3, 22, 23, 16, 57, 58839)

In [2]: Entry.select(fn.MAX(Entry.created_date)).scalar(convert=False)
Out[2]: u'2016-03-22 23:16:57.058839'
```

The reason, I believe, that you aren't seeing similar behavior with Postgresql is because `psycopg2`, the Postgresql/Python library, is doing the conversion before the data gets to peewee.

I think you're best off just hucking some `datetime.datetime()` objects in there, or implementing a custom field that converts timestamps into datetime objects (and remembering to pass `convert=True` when using scalar).
Issue: Id field is not created with AUTOINCREMENT
Author: mamcx
State: closed
Created: 2016-05-11 15:38:37

I have a normal model and the tables have the ID marked as not AUTOINCREMENT.

This is the code:

``` python
class BaseModel(Model):
    class Meta:
        database = db

class Cobro(BaseModel):
    nombre = CharField()
```

``` sql
CREATE TABLE "cobro"(
    "id"     Integer NOT NULL PRIMARY KEY,
    "nombre" Text NOT NULL )
```


It appears you are using Sqlite. Please read http://sqlite.org/autoinc.html for an explanation why this is not the default behavior. If you want autoincrement with Sqlite, you can use [PrimaryKeyAutoIncrementField](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#PrimaryKeyAutoIncrementField) in `playhouse.sqlite_ext`.
Issue: pwiz fails to generate correct code: UnknownField
Author: james-lawrence
State: closed
Created: 2016-05-10 17:58:03

```
createdb
psql <<EOF
CREATE TABLE example (
  address inet DEFAULT '0.0.0.0'::inet NOT NULL PRIMARY KEY
);
EOF
python -m pwiz -e postgresql -u $USER $USER
```

generates the following code:

```
class UnknownField(object):
    pass

class BaseModel(Model):
    class Meta:
        database = database

class Example(BaseModel):
    address = UnknownField(primary_key=True)  # inet
    class Meta:
        db_table = 'example'
```

UnknownField blows up because it expects named args and it has no initializer.
it should be:

```
class UnknownField(object):
    def __init__(self, **kwargs):
        pass
```
Issue: pwiz fails to generate correct code: missing extension import.
Author: james-lawrence
State: closed
Created: 2016-05-10 17:50:58

if a table contains a json field, NameError: name 'JSONField' is not defined is raised.
pwiz should just include the import, it shouldn't generate code that doesn't work.

```
createdb
psql <<EOF
CREATE TABLE example (
  field1 json
);
EOF
python -m pwiz -e postgresql -u $USER $USER
```

generated the following code:

```
from peewee import *

database = PostgresqlDatabase('user', **{'user': 'user'})

class UnknownField(object):
    pass

class BaseModel(Model):
    class Meta:
        database = database

class Example(BaseModel):
    field1 = JSONField(null=True)

    class Meta:
        db_table = 'example'
```

generated code blows up on the JSONField()


Not the author but I'll answer anyways.

`JSONField` is contained in `playhouse.postgres_ext` so you need another import, i.e

``` python
from peewee import *
from playhouse.postgres_ext import JSONField # <----------

database = PostgresqlDatabase('user', **{'user': 'user'})

class UnknownField(object):
    pass

class BaseModel(Model):
    class Meta:
        database = database

class Example(BaseModel):
    field1 = JSONField(null=True)

    class Meta:
        db_table = 'example'
```

See http://peewee.readthedocs.io/en/latest/peewee/playhouse.html#postgresql-extensions


I'm well aware of the fix and how to fix it. my point is if pwiz is going to generate this code it should generate it correctly. i.e.) it includes the required import automatically. I shouldn't have to modify the code after.


Oh, sorry, didn't realize that all of it was generated.
Issue: How to use "select distinct binary  content from table;" in peewee?
Author: ladder1984
State: closed
Created: 2016-05-10 07:02:48

How to use "select distinct binary  content from table;" in peewee? I want to use "binary" when "select distinct " .
What's more,I use peewee 2.6.4.


You can do a couple things, but think the simplest would be something like:

``` python
from peewee import *

MyModel.select(Clause(SQL('distinct binary'), MyModel.content))

# select distinct binary t1.content from ...
```

You may need to attach an alias to the clause so peewee knows what attribute the value belongs to:

`, MyModel.content).alias('content')`
Issue: peewee-versioned 0.1
Author: kramer65
State: closed
Created: 2016-05-06 06:22:11

Hi @coleifer - after using the code in our own website for a little while I've now released the code for peewee versioned on my github account here: https://github.com/kramer65/peewee-versioned

It would be great if you can have a look at it and possibly test it. I'd appreciate feedback on how to usage works now and how we can make that more intuitive.

And of course feel free to link to it from the peewee docs. Other people might find this useful as well. :-)


Looks nice! I'll leave it to Google to help people find your project but if someone should ask about similar work I'll certainly keep this in mind. I'm kind of curious how foreign keys / referential integrity works when you delete a model but older versions of another table relate to the now-deleted object. There are maybe some edge cases? It also looks like there are gaps in the valid_from / valid_until that probably shouldn't be there... Other than that, though looks good!
Issue: Extra Parenthesis
Author: stanep
State: closed
Created: 2016-05-05 14:07:33

Hi if I have query like this
qr = Cwle.select(Cwle.WaitlistEntryID).filter(
                  (Cwle.WaitlistEntryID == 1) & (Cwle.WaitlistEntryID == 4)
                 )

print qr , will give you 

SELECT CWLE.WaitlistEntryID FROM WCPRDDB1.dbo.CardiacWaitListEntry AS CWLE WHERE ((CWLE.WaitlistEntryID = ?) AND (CWLE.WaitlistEntryID = ?)) [1, 4]

Is it possible to remove outer Parenthesis?

SELECT CWLE.WaitlistEntryID FROM WCPRDDB1.dbo.CardiacWaitListEntry AS CWLE WHERE (CWLE.WaitlistEntryID = ? AND CWLE.WaitlistEntryID = ?) [1, 4]

Reason being that sometimes I want to get query from peewee and run it in db management tool, all
these extra Parenthesis makes your eyes bleeds :)


Sorry, you'll need to write a tool to strip them out or extend the QueryCompiler class.
Issue: Abstract Model database metadata inheritance doesn't work
Author: erwagasore
State: closed
Created: 2016-05-04 15:21:57

``` python
db = PostgresqlDatabase(...)

class BaseModel(Model):
    created_at = DateTimeField(default=datetime.now)
    modified_at = DateTimeField(default=datetime.now)

    class Meta:
        abstract = True
        db_table_func = lambda model_cls: tableize(model_cls.__name__)
        database = db

class Foo(BaseModel):
    name = CharField()

foo = Foo(name="Foo Bar")
foo.name                            #-> Foo Bar
foo._meta.db_table            #-> foos
foo._meta.database           #-> <peewee.SqliteDatabase object at 0x1012e64a8>
```

Inheritance from the abstract model works on field and other _meta arguments but database.
when I call `create_tables` for all model i found that an sqlite db have been created called `peewee.db`


First off, where did you get the `meta.abstract` thing? That is a django orm thing, not peewee.


Oh. I thought peewee have similar implementation. My mistake.
Does peewee have something like this?


I should close this issue then


Also the code works fine for me:

``` python

In [5]: from peewee import *

In [6]: db = SqliteDatabase('foo.db')

In [7]: db.database
Out[7]: 'foo.db'

In [8]: class BaseModel(Model):
    timestamp = DateTimeField()
    class Meta:
        database = db
        db_table_func = lambda model_cls: model_cls.__name__.title()
   ...:         

In [9]: class TestModel(BaseModel):
   ...:     name = CharField()
   ...:     

In [10]: tm = TestModel(name='test')

In [11]: tm._meta.database.database
Out[11]: 'foo.db'

In [12]: tm._meta.db_table
Out[12]: 'Testmodel'
```


I am using `from playhouse.flask_utils import FlaskDB` and `from playhouse.signals import Model` I guess that mix is the one causing issues 


It is trick if you want to benefit from both playhouse flask utils `FlaskDB` and also benefit from signals


@coleifer I found out the cause of such wild behaviour.

The cause was that I was using `playhouse.signals.Model`with `playhouse.flask_utils.FlaskDB`.

I solved it by using both

``` python
from playhouse.flask_utils import FlaskDB
from playhouse.signals import Model

db = FlaskDB(app, ....)

class BaseModel(db.Model, Model):
    class Meta:
        database = db.database

class Foo(BaseModel):
    name = CharField()
...
```

I don't know if this is the sane way to do it but it seems to work


Yes, FlaskDB is a wrapper and you still need to point at the underlying peewee db


Cool. Thanks


@coleifer, `BaseModel` from your example will also generate a table in db. Do we have any option to specify that for a specific model we don't want a table?

Only if you call `BasModel.create_table()` or pass it to database.create_tables(), so I'm not sure I see the problem.
Issue: RetryOperationalError should raise peewee.OperationalError
Author: yami
State: closed
Created: 2016-05-04 06:28:51

With current RetryOperationalError implementation, for mysql, a second operation error will cause _mysql_exceptions.OperationalError being re-raised. I'd expect peewee.OperationalError in this case.

``` python
class RetryOperationalError(object):
    def execute_sql(self, sql, params=None, require_commit=True):
        try:
            cursor = super(RetryOperationalError, self).execute_sql(
                sql, params, require_commit)
        except OperationalError:
            if not self.is_closed():
                self.close()
            cursor = self.get_cursor()
            cursor.execute(sql, params or ())
            if require_commit and self.get_autocommit():
                self.commit()
        return cursor
```


What will happen, if there are several statements in one transaction, and in the middle of execution, an operation error is raised?


> What will happen, if there are several statements in one transaction, and in the middle of execution, an operation error is raised?

You can read the code to find out!

Its actually a bit complex so I'll try to explain. If you are not in a transaction, have autocommit enabled (on by default), and autorollback enabled (**off** by default), then the txn will be rolled back and the exception raised for your application to handle.

When you enter a transaction, peewee tells the database to disable autocommit, so the automatic rollback will not happen in `execute_sql()`. If an unhandled exception does bubble up to the `transaction` object, though, it will rollback before raising the exception to your code.

So if you're using the `RetryOperationalError`mixin, the first `OperationalError` is caught before it bubbles up to the txn so nothing happens. If the retry fails, though, you will get a rollback and an exception.


should have commit or rollback before to raise exception?

``` python
  def execute_sql(self, sql, params=None, require_commit=True):
        try:
            cursor = super(RetryOperationalError, self).execute_sql(
                sql, params, require_commit)
        except OperationalError:
            if not self.is_closed():
                self.close()
            with self.exception_wrapper():
                cursor = self.get_cursor()
                try:
                    cursor.execute(sql, params or ())
                except Exception:
                    if self.get_autocommit() and self.autorollback:
                        self.rollback()
                    raise
                else:
                    if require_commit and self.get_autocommit():
                        self.commit()
        return cursor
```


@mathisonqin -- I'm sorry but I'm not following your comment. Do you mind clarifying?
Issue: Is there an API on ForeignKeyField to generate dynamically related_name
Author: erwagasore
State: closed
Created: 2016-04-29 15:55:17

I would like to create a base model with some dynamically generated foreignkeyfield `related_name`. I was hoping for a way to deal with it without touching the model constructor. Same thing you did with `db_table` by providing `db_table_func`

``` python
class AwareModel(Model):
    created_by = ForeignKeyField(User, related_name_func = lambda model_cls: ...)
    modified_by = ForeignKeyField(User, related_name_func = lambda model_cls: ...)

...

class Restaurant(AwareModel):
    name = CharField()

...

# supposed there is a user called foo then I would be able to call the restaurants from the user
foo.created_restaurants   # -> A list of restaurants foo created
foo.modified_restaurants   # -> A list of restaurants foo modified


class Comment(AwareModel):
    content = TextField()

...

# supposed there is a user called foo then I would be able to call the restaurants from the user
foo.created_comments   # -> A list of comments foo created
foo.modified_comments   # -> A list of comments foo modified
```


I would question the usefulness of dynamically generated backrefs, but whatever.

Pushed a commit 5f7078929b8731b74ab2bf0dede28d7bc103c8e1 that allows foreign key related_name init param to be a callable that accepts the field as it's only param.
Issue: Dynamically generate db_table name
Author: erwagasore
State: closed
Created: 2016-04-29 08:36:51

I want to create an abstract base model which will generate db_table name dynamically.
Below is an illustration of what I need to accomplish.

``` python
from inflection import tableize
...

class User(Model):
    pass

...


class BaseModel(Model):
    class Meta:
        abstract = True
        # make db_table dynamically generated based on the class name
        # db_table = tableize(cls.__name__)  ------  'RoomService' -> 'room_services'

...


class RoomService(BaseModel):
    name = CharField()

foo = RoomService.create(name='Foo Bar')
# foo._meta.db_table  --->  room_services
```

Is this a thing that can be done in Peewee? Thanks


You could do something like this:

``` python
import peewee as p
from inflection import tableize

class BaseModelMeta(p.BaseModel):
    def __new__(cls, name, bases, attrs):
        cls = super(BaseModel, cls).__new__(cls, name, bases, attrs)

        # overwrite db_table name
        cls._meta.db_table = tableize(name)

        return cls

class BaseModel(p.with_metaclass(BaseModelMeta, p.Model), p.Model):
    # the second p.Model is optional, it gives me autocompletion in pyCharm
    pass
```


There's an API for this:

``` python
class MyModel(Model):
    class Meta:
        db_table_func = lambda model_cls: ...
```


Listed a little below this section heading: http://docs.peewee-orm.com/en/latest/peewee/models.html#model-options-and-table-metadata


Here I was, thinking I knew the documentation by heart... Back to the school-books for me then!
Issue: Add configuration check for sqlite
Author: M157q
State: closed
Created: 2016-04-28 15:21:15

Some operating systems don't have sqlite3 module within python3.
without this commit, they will get unclear error message like this:

```
File "/usr/local/lib/python3.4/site-packages/peewee.py", line 3371, in
connect
    **self.connect_kwargs)
  File "/usr/local/lib/python3.4/site-packages/peewee.py", line 3609, in
_connect
    conn = sqlite3.connect(database, **kwargs)
AttributeError: 'NoneType' object has no attribute 'connect'
```

Those users who encounter this ImproperlyConfiguredError
may have to install sqlite dependencies and recompile their python3


Thanks, good idea!
Issue: Many to many relationship within itself
Author: ghost
State: closed
Created: 2016-04-26 20:28:29

Not sure if this is an issue rather than a question, but here it goes..

Given a model A, and  a model B (a many to many table)

``` python
class B(Model):

    original_A = ForeignKeyField(
        A,
        related_name='AB'
    )
    related_A = ForeignKeyField( 
        A,
        related_name='BA'
    )

    class Meta:
        database = db
```

The 2 foreign keys, as you can see, relate to the same model.
The tricky part gets when you try to query it,

The query is: (C is something relating to A)

``` python
query = models.A.select(
                A,
                B,
                C
            ).join(
                models.C
            ).switch(            ##
                models.A
            ).join(
                models.B
            ).join(
                models.A
            ).join(
                models.C
            )
```

So, how can i get rid of the ambiguity and say that the first join of A with model B neets to relate to the first column of B, and the second join of B with A needs to relate to the second column of B?

In the end what i want to get is:

``` python
for a in query:
    #do something with a
    for c in a:
        #do something with its related c
    for b in a:
        for ba in b:
            #do something with its relations with itself
            for c in ba:
                #do something with its related c of its related self..
```

kind regards,
Thomas


and throw in an .aggregate_rows() at the end, though i'm not sure how it'd respond with so many joins.


@Thomaxo -- what you want is to probably use a combination of model aliases and explicit join predicates.

First off, it might make better sense to query the "B" table (the junction table). For example:

``` python
A2 = A.alias()
C2 = C.alias()
query = (B
         .select(B, A, A2, C, C2)
         .join(A, on=(B.original_A == A.id))
         .join(C, on=(A.c == C.id))
         .switch(B)
         .join(A2, on=(B.related_A == A2.id).alias('related_A'))
         .join(C2, on=(A2.c == C2.id).alias('c')))

for b in query:
    print b.original_A, b.original_A.c
    print b.related_A, b.related_A.c
```


If you believe you've found an issue with Peewee, please open an issue. Otherwise, if you're still stuck, I'd suggest the mailing list or stackoverflow.


Thanks for the help! :)
Issue: Sqlite datetime with timezone is not converted into datetime
Author: goir
State: closed
Created: 2016-04-20 07:16:21

If you store a timezone aware datetime in sqlite the selected value is a unicode and not a datetime object.

```
import datetime

import pytz
from peewee import SqliteDatabase, PrimaryKeyField, DateTimeField, Model

db = SqliteDatabase('test.db')


class ModelA(Model):
    id = PrimaryKeyField()
    date = DateTimeField()

    class Meta(object):
        database = db

ModelA.create_table(True)

ModelA.create(date=datetime.datetime.now(pytz.UTC))
ModelA.create(date=datetime.datetime.now())

for a in ModelA.select():
    print a.date, type(a.date)
```

which returns

```
2016-04-20 07:06:18.021696+00:00 <type 'unicode'>
2016-04-20 09:06:18.420155 <type 'datetime.datetime'>
```

the unicode should be a datetime object with timezone set.


SQLite will store timezone info, but I'm not sure what facilities it provides for sorting when you've got a mixture of timezones (without involving various date functions). You'll probably want to read up on it here: https://www.sqlite.org/lang_datefunc.html -- It might be more convenient to convert to UTC in your application before passing ot the database.

That said, it does seem like peewee should handle the timezone. For now you can subclass the `DateTimeField` and override the formats class attribute.

I'll look into a fix.


I agree, its not really an issue, more a feature request to have a timezone aware datetime field.


I don't believe I will implement this. If you want to register your own adapter at the driver level, or if you want to implement your own TZ-aware sqlite datetime-field, then I think that'd be the best course. A general purpose solution, though, I'm going to pass on.