akumoth/peewee-issues huggingface dataset:
Issue: pylint doesn't see some peewee Model fields
Author: richey-v
State: closed
Created: 2015-01-14 23:58:41

I'm using peewee with Flask.

I have an existing MySQL database and used pwiz to generate a decent model of it.  Functionally, everything is working fine, but pylint still complains about a Model field being not detected.

Here is the SQL:

``` SQL
CREATE TABLE name_instance (
    id                    INT   NOT NULL AUTO_INCREMENT,
    name_id               INT   NOT NULL,
    alias_id              INT   UNIQUE,
    PRIMARY KEY (id),
    FOREIGN KEY (name_id)   REFERENCES name_variation (id)  ON DELETE CASCADE,
    FOREIGN KEY (alias_id)  REFERENCES name_variation (id)  ON DELETE CASCADE
);
```

Here is the pwiz-generated class:

``` python
class NameInstance(BaseModel):
    alias = ForeignKeyField(db_column='alias_id',
                            null=True,
                            rel_model=NameVariation,
                            related_name='aliases',
                            to_field='id',
                            unique=True)
    name = ForeignKeyField(db_column='name_id',
                           rel_model=NameVariation,
                           related_name='names',
                           to_field='id')

    class Meta:
        db_table = 'name_instance'
```

In Flask code, I can do this without pylint complaining:

``` python
   NameInstance.get(NameInstance.name == name_inst_id)
```

but pylint complains about "id" not defined in the following line:

``` python
   NameInstance.get(NameInstance.id == name_inst_id
```


Can you please format your issue? You can edit it and apply markdown style.


If a model has an auto-increment integer primary key named `id`, the convention is to omit defining it explicitly. This convention comes from Django and I like it. If you want to fix it, just add this field definition to your models:

``` python
id = PrimaryKeyField()
```

Also I see in your schema the foreign keys are `NOT NULL` but in peewee they are marked `null=True` -- you might want to remove that.
Issue: Custom field not getting parsed using alias
Author: BusyJay
State: closed
Created: 2015-01-08 19:17:42

Hi, I have a custom field definition like:

```
class CustomField(peewee.Field):
    def coerce(self, value):
        return "parsed"

class A(peewee.Model):
    f = CustomField()
```

When I parse custom value to Models directly, it works fine.

```
>>> A.select().where(A.f == 20)
<class '__main__.A'> SELECT "t1"."id", "t1"."f" FROM "a" AS t1 WHERE ("t1"."f" = ?) ['parsed']
```

But, when I use alias, it won't parse the value for me.

```
>>> a = A.alias()
>>> A.select().join(a, on=(A.f == a.f)).where(a.f == 20)
<class '__main__.A'> SELECT "t1"."id", "t1"."f" FROM "a" AS t1 INNER JOIN "a" AS t2 ON ("t1"."f" = "t2"."f") WHERE ("t2"."f" = ?) [20]
```

What I am expecting is:

```
<class '__main__.A'> SELECT "t1"."id", "t1"."f" FROM "a" AS t1 INNER JOIN "a" AS t2 ON ("t1"."f" = "t2"."f") WHERE ("t2"."f" = ?) ['parsed']
```


When will be next release?


I'll create one soon.
Issue: Add jsonb field support for postgres
Author: extesy
State: closed
Created: 2015-01-07 16:39:18

PostgreSQL 9.4 now supports binary json field (jsonb), the documentation is here: http://www.postgresql.org/docs/9.4/static/datatype-json.html


Yes! I'm very interested in adding this. I currently have 9.3.5 installed so I'll need to get 9.4 running in order to start working on this.


0f48020cf2604966aa77dcd57044b738cc7f44de


I've made a start at implementing this. I'll put together some docs shortly.


:thumbsup:


Any update on this issue @coleifer ???


Support was merged in.
Issue: Union with Meta "order_by" clause produces incorrect SQL query
Author: albatros69
State: closed
Created: 2015-01-05 22:55:36

Hello,

I've tried the following (rather complex I admit) but ended up to an incorrect SQL statement (at least with SQLite). Commenting the "order_by" in the Meta class is a workaround.

```
class Type(Model):                                                          
     [...]                                                 

     class Meta:                                                                 
          db_table = 'types'                                                      
          order_by = ('name', )                                                   

q_orders = cls.select(Type, Order.quantity).join(Wine).join(Order).\ 
            where(Type.owner == login.current_user.id).group_by(Type.id)       
q_changes = cls.select(Type, fn.sum(ChangeLog.change).alias('quantity')).\
            join(Wine).join(Order).join(ChangeLog).\                        
            where(Type.owner == login.current_user.id).group_by(Type.id)       
print (q_orders | q_changes)                                            
```

Result:

``` sql
      SELECT "t2"."id", "t2"."html_color", "t2"."name", "t2"."owner_id", "t4"."quantity" 
FROM "types" AS t2 
INNER JOIN "wines" AS t3 
ON ("t2"."id" = "t3"."type_id") 
INNER JOIN "orders" AS t4 ON ("t3"."id" = "t4"."wine_id") 
WHERE ("t2"."owner_id" = ?) 
GROUP BY "t2"."id" 
ORDER BY "t2"."name" ASC 

UNION 

SELECT "t2"."id", "t2"."html_color", "t2"."name", "t2"."owner_id", sum("t5"."change") AS quantity 
FROM "types" AS t2 
INNER JOIN "wines" AS t3 ON ("t2"."id" = "t3"."type_id") 
INNER JOIN "orders" AS t4 ON ("t3"."id" = "t4"."wine_id") 
INNER JOIN "changelog" AS t5 ON ("t4"."id" = "t5"."order_id") 
WHERE ("t2"."owner_id" = ?) 
GROUP BY "t2"."id" 
ORDER BY "t2"."name" ASC
```

Which leads to the following SQL error: `Error: ORDER BY clause should come after UNION not before`


This is one of many reasons why default ordering is a bad idea. I regret adding this functionality, but it's too late to remove it now.

At any rate, you can clear the existing order by clauses on your sub-queries. That should fix the problem:

``` python
q_orders = cls.select(Type, Order.quantity).join(Wine).join(Order).\
            where(Type.owner == login.current_user.id).group_by(Type.id).\
            order_by()
q_changes = cls.select(Type, fn.sum(ChangeLog.change).alias('quantity')).\
            join(Wine).join(Order).join(ChangeLog).\
            where(Type.owner == login.current_user.id).group_by(Type.id).\
            order_by()
```
Issue: [fix] TypeError: host is an invalid keyword argument for this function
Author: ekatsah
State: closed
Created: 2015-01-05 11:33:13

Try to fix this:

```
self = <peewee.SqliteDatabase object at 0x7f79bbfcf950>, database = ':memory:'
kwargs = {'host': file.db'}

def _connect(self, database, **kwargs):
>       conn = sqlite3.connect(database, **kwargs)
E TypeError: 'host' is an invalid keyword argument for this function
```


I'm confused -- why is `host` getting used?


Hum i read the doc too rapidly and confused relatif/absolute path. please discard this PR. sorry for the noise.

Was trying to fix this kind of error

```
self = <peewee.SqliteDatabase object at 0x7fc15db82c90>
database = 'tmp/file.db', kwargs = {}

def _connect(self, database, **kwargs):
>       conn = sqlite3.connect(database, **kwargs)
E OperationalError: unable to open database file
```

but the doc is right. furthermore, this fix doesnt cover the use case (absolute path...)
Issue: Foreign Key Issue
Author: ghost
State: closed
Created: 2015-01-02 16:59:35

I just encountered this issue. Not sure if it had already been reported.

I have my models defined as follows

```
class BaseModel(Model):
    class Meta:
        database = db

class Login(BaseModel):
    username = TextField(unique=True)
    password = TextField()

class Profile(BaseModel):
    dname = TextField()
    about = TextField()
    photo = TextField()

class User(BaseModel):
    login = ForeignKeyField(Login, on_delete='CASCADE')
    profile = ForeignKeyField(Profile, null=True, 
                              on_delete='CASCADE')

```

When I call `user.delete_instance()` the record from user table is deleted, but record from login table persists. It is wrong right?


What database engine are you using? That's definitely a bit odd.


If you're using SQLite, you will need to execute `PRAGMA foreign_keys=on;` when you open a connection in order for this to work correctly.


Thank you. I am using SQLite, and this seems to be the solution. :+1:


The problem persists.

Tried executing `PRAGMA foreign_keys=on;` using Flasks `before_request` decorator like this

```
@app.before_request
def before_request():
    g.db = db
    g.db.execute_sql('PRAGMA foreign_keys=on')
    g.db.connect()
```

The problem still persists.


You just need to swap so you execute the pragma statement **on** the new connection.

``` python
    g.db = db
    g.db.connect()
    g.db.execute_sql('PRAGMA foreign_keys=on')
```

Alternatively you could subclass `SqliteDatabase`:

``` python

class SqliteFKDatabase(SqliteDatabase):
    def _add_conn_hooks(self, conn):
        super(SqliteFKDatabase, self)._add_conn_hooks(conn)
        self.execute_sql('PRAGMA foreign_keys=on;')
```


This seems not like a SQLite specific issue. Tried using MySQL too. The problem still persists. 
Wondering wether my model is right. Also `delete_instance()` is enough right?


``` python
from peewee import *

db = SqliteDatabase(':memory:')
db.execute_sql('pragma foreign_keys=on;')

class Base(Model):
    class Meta:
        database = db

class A(Base):
    data = IntegerField(default=0)

class B(Base):
    a = ForeignKeyField(A, on_delete='CASCADE')

db.create_tables([A, B])

a1 = A.create()
a2 = A.create()
for a in [a1, a2]:
    for i in range(3):
        B.create(a=a)

print B.select().count()
a1.delete_instance()
print B.select().count()
```

Prints:

```
6
3
```


Thank you. I had got the concept wrong. 
Every thing is fine now :+1:
Issue: pwiz issue with composite primary keys
Author: dkorduban
State: closed
Created: 2014-12-31 11:06:45

It generates something like

```
class A(BaseModel):
    id1 = IntegerField(primary_key=True)
    id2 = IntegerField(primary_key=True)

    class Meta:
        db_table = 'a'
        primary_key = CompositeKey('id1', 'id2')
```

which in turn leads to

```
ValueError: primary key is overdetermined.
```


You should not specify `primary_key=True` if you are specifying a `CompositeKey`. Instead write:

``` python
class A(BaseModel):
    id1 = IntegerField()
    id2 = IntegerField()

    class Meta:
        db_table = 'a'
        primary_key = CompositeKey('id1', 'id2')
```


Oh shit, should have read this more closely. You're right.
Issue: PyPy count issue
Author: stas
State: closed
Created: 2014-12-29 13:04:25

I was upgrading a package and some of our tests on PyPy failed.
The backtrace goes down to:

```
peewee-2.4.5-py2.7.egg/peewee.py", line 2433, in _aggregate
    aggregation = fn.Count(SQL('*'))
AttributeError: 'builtin_function' object has no attribute 'Count'

```

Interesting fact, that running the same tests separately works. Only running it with the whole suite, at some point starts failing.
Any ideas what could cause this?

Thanks in advance.


Closing this, looks like a https://github.com/spulec/freezegun issue with PyPy.
Issue: Database connection not refreshed after wait_timeout
Author: arunisaac
State: closed
Created: 2014-12-28 18:19:11

I'm using peewee with a MySQL database with pymysql being the database driver. The wait_timeout of my mysql database is set to the default value of 8 hours.

When my application remains idle for 8 hours, the database connection times out, and any further requests throw MySQL errors 2013 (Lost connection to MySQL server during query) and 2006 (MySQL server has gone away). Similar behaviour also happens with PooledMySQLDatabase connections.

Is this behaviour normal? Should I do something to make peewee to reconnect timed out connections, or is pymysql supposed to take care of this?


You need to close connections when you're done with them. Neither pymysql nor peewee will detect the 2013/2006 error and re-connect automatically.


If you're using the connection pool, then when you close a connection it will be returned to the pool and recycled until it reaches it's stale timeout (specified when initializing the pool).

``` python

db = PooledMySQLDatabase('my_db', stale_timeout=300)  # Consider a conn stale after 5 mins.

db.connect()
# do some stuff.
db.close()  # Return connection to the pool.
```


Ok. I didn't realize I had to manually connect and close database connections. That makes sense. Thanks a lot!

I do realize this is a very newbie kind of problem, but maybe you could put some more emphasis on this in the documentation - possibly in the quickstart section.

Anyways, thanks for the great ORM! Your work is very much appreciated!


Yeah, while I have added some documentation to this effect, it could certainly stand to be improved. You're not the first person to find this confusing, so I'll make a point to clean up the docs.


![s1419837109 48](https://cloud.githubusercontent.com/assets/119974/5566767/fd8aa9a2-8ef7-11e4-96e3-e1e505e3dde8.png)


Explicitly putting the db.connect() and db.close() statements in some example code might help make the point clear. I noticed that this is shown in the "Example app" page in the documentation. But, I hadn't read it until now, because I use cherrypy and I thought the "Example app" page was flask specific and not relevant to me. I was mostly going through the "API Reference", "Managing your Database" and "Quickstart" sections of the documentation to try and figure out what had happened.


good idea
Issue: indexes not working: ValueError: too many values to unpack (expected 2)
Author: kshade
State: closed
Created: 2014-12-28 17:15:33

I'm writing a small SQLite-based application with peewee 2.4.5 for which I need to import data from another souce that, sadly, doesn't supply unique IDs. Therefore I want to have my own, internal primary keys and a constraint that makes sure I don't import the same row multiple times. I'm trying to use an unique index to do so:

``` python
#!/bin/env python3
# -*- coding: utf-8 -*-
import peewee as pw

DATABASE = pw.SqliteDatabase('testcase.sqlite3')

class BaseModel(pw.Model):
    class Meta:
        database = DATABASE


class Employee(BaseModel):
    eid = pw.PrimaryKeyField()
    gesamtbrutto = pw.IntegerField(null=True)
    vorname = pw.TextField(null=True)
    nachname = pw.TextField(null=True)

    class Meta:
        indexes = ((('vorname', 'nachname', 'gesamtbrutto'), True))

if __name__ == "__main__":
    DATABASE.connect()
    DATABASE.create_tables([Employee], safe=True)
```

Sadly this doesn't work:

```
$ rm testcase.sqlite3
$ ./testcase.py 
Traceback (most recent call last):
  File "./testcase.py", line 23, in <module>
    DATABASE.create_tables([Employee], safe=True)
  File "/usr/lib64/python3.3/site-packages/peewee.py", line 2915, in create_tables
    create_model_tables(models, fail_silently=safe)
  File "/usr/lib64/python3.3/site-packages/peewee.py", line 3983, in create_model_tables
    m.create_table(**create_table_kwargs)
  File "/usr/lib64/python3.3/site-packages/peewee.py", line 3740, in create_table
    cls._create_indexes()
  File "/usr/lib64/python3.3/site-packages/peewee.py", line 3763, in _create_indexes
    for fields, unique in cls._meta.indexes:
ValueError: too many values to unpack (expected 2)
$ sqlite3 testcase.sqlite3
SQLite version 3.8.7.4 2014-12-09 01:34:36
Enter ".help" for usage hints.
sqlite> .dump employee
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE "employee" ("eid" INTEGER NOT NULL PRIMARY KEY, "gesamtbrutto" INTEGER, "vorname" TEXT, "nachname" TEXT);
COMMIT;
```

Am I missing something here? Is this feature not available in SQLite?


``` python
    class Meta:
        indexes = (
            (('vorname', 'nachname', 'gesamtbrutto'), True),  # Add trailing comma.
        )
```
Issue: Docs are generated from master instead of latest stable
Author: timovwb
State: closed
Created: 2014-12-24 15:46:46

It looks like the documentation is generated against master instead of the latest stable release. I was looking at implementing ManyToMany and noticed I couldn't import the `ManyToManyField` which is mentioned in the docs. A quick look at the latest commits show that this is added after the 2.4.5 release. It's a bit confusing as the page title of the documentation mentions that 2.4.5 version.

Mentioned at these two pages:
http://peewee.readthedocs.org/en/latest/peewee/querying.html#manytomanyfield
http://peewee.readthedocs.org/en/latest/peewee/playhouse.html#ManyToManyField


There's now a `stable` branch with corresponding docs. I will keep the `stable` branch pinned to the latest tag:

http://docs.peewee-orm.com/en/stable/
Issue: Support for Expression-based joins when using aggregate_rows()
Author: arrowgamer
State: closed
Created: 2014-12-20 21:21:35

Joining on an Expression when using aggregate_rows() would cause an error because rel_for_model only handles Fields.

Consider the following data models:

``` python
class Organization(BaseModel):
    name = CharField()


class Review(BaseModel):
    rating = IntegerField()
    organization = ForeignKeyField(Organization, related_name='reviews')
```

I want to query for an organization by ID and eagerly load its reviews that are higher than 3.

``` python
(Organization
 .select(Organization, Review)
 .join(Review, JOIN_LEFT_OUTER,
        on=((Review.organization == Organization.id) & (Review.rating > 3)).alias('organization'))
 .where(Organization.id == 1)
 .aggregate_rows())[0]
```

Note the usage of [0] because of #483. The above would fail because rel_for_model does not support Expressions. I am doing the filter on the join instead of the where, because the query should still pull organization data even if there are no reviews above the rating of 3.


Thanks a lot. I'm sorry for all the pull requests and stuff. These are just the issues I've found thus far in using your ORM in a personal project of mine, and I thought it'd be nice to share them so others don't run into the same =)


No, I appreciate it! Thanks for your help!


Sure! =)
Issue: Using .get() with .aggregate_rows()
Author: arrowgamer
State: closed
Created: 2014-12-20 21:08:30

Consider the following data models:

``` python
class Organization(BaseModel):
    name = CharField()


class Review(BaseModel):
    rating = IntegerField()
    organization = ForeignKeyField(Organization, related_name='reviews')
```

If I wanted to get an organization by its ID and eagerly load its reviews, the LIMIT 1 that gets applied to the query due to .get() prevents it from getting any more than the first review.

``` python
(Organization
 .select(Organization, Review)
 .join(Review, JOIN_LEFT_OUTER)
 .where(Organization.id == 1)
 .aggregate_rows()).get()
```

Is this expected functionality? Would this need to be done using prefetch()? Or is there a different way of structuring this query?


Refs #480


Your options proposed there do make sense! Performing a separate query isn't that big of an issue given that the usage of get() implies there will only be one parent object. However, in the case that you're joining on many different tables, I suppose you'd need a separate query for each of them, or use the [0] option as I did in #484.

Please note that #484 is not dependent on single row selects, however, as it can apply to selecting all rows with an Expression-based join.
Issue: Support for filtering by joined table outside of select
Author: arrowgamer
State: closed
Created: 2014-12-20 09:35:34

Filtering by a joined table that is not part of the select would cause a KeyError when using aggregate_rows.

Consider the following example models:

``` python
class Organization(Model):
    name = CharField()


class Job(Model):
    name = CharField()
    organization = ForeignKeyField(Organization)


class Employee(Model):
    name = CharField()
    organization = ForeignKeyField(Organization)
    job = ForeignKeyField(Job)
```

Setup some basic records like so:

``` python
testing = Organization.create(name='Testing Co.')
tester = Job.create(name='Tester', organization=testing)
Employee.create(name='Tester Guy', organization=testing, job=tester)
```

The below throws an unhandled KeyError because the second join that is used to filter is not included in the select. The intent of the query below is to eagerly load an organization and all its jobs based on an employee's name, but not to load the employee's data.

``` python
(Organization
 .select(Organization, Job)
 .join(Job)
 .switch(Organization)
 .join(Employee)
 .where(Employee.name == 'Tester Guy')
 .aggregate_rows()
 .get())
```

Adding the try/except after line 2036 fixes this.

The below throws an unhandled KeyError because the second join that is used to filter is not included in the select. The intent of the query below is to eagerly load an employee and all its jobs based on an organization's name, but not to load the organization's data.

``` python
(Employee
 .select(Employee, Job)
 .join(Job)
 .switch(Employee)
 .join(Organization)
 .where(Organization.name == 'Testing Co.')
 .aggregate_rows()
 .get())
```

Adding the try/except after line 2020 fixes this.


I fixed this with a slightly smaller change: 4c38abd5ea1c0d05226e181fced3895ada8ee73d


I took a look. Awesome!
Issue: ForeignKey related_name problem
Author: tigrus
State: closed
Created: 2014-12-19 22:02:54

We have following models: 

```
class ResUsers(BaseModel):
    #gave up here... it reference to self, but have same error
    create_uid = peewee.IntegerField()

class ProductPriceType(BaseModel):
    class Meta:
        managed = False
        db_table = 'product_price_type'

    create_uid = peewee.ForeignKeyField(ResUsers, null=True, related_name="create_uid")
```

When I'm using this models, I have following error:
    AttributeError: Foreign key: productpricetype.create_uid related name "create_uid" collision with model field of the same name.

Temporary solution - usage of IntegerFields.
How this part of code should be designed with ForeignKeys? We use existing legacy db scheme. 


Related names are "back references", a handy way of accessing a collection of objects relating _to_ the given instance. For example say you have user and tweet models. You would say a tweet has one user, while a user has many tweets. That gives you a foreign key named user and a related name of tweets:

``` python
class User(Model):
    username = CharField()

class Tweet(Model):
    user = ForeignKeyField(User, related_name='tweets')
    content = TextField()
```

This will patch a new attribute onto the user model named `tweets`. If the `User` already had a field named `tweets`, though, then peewee would not like it because the related name attribute it's trying to add is taken.

That's one part of the issue. To fix this, I would suggest calling your related_name "res_users":

``` python
class ResUsers(BaseModel):
    #gave up here... it reference to self, but have same error
    create_uid = peewee.IntegerField()

class ProductPriceType(BaseModel):
    class Meta:
        managed = False
        db_table = 'product_price_type'

    create_uid = peewee.ForeignKeyField(ResUsers, null=True, related_name="res_users")
```

To create a self-referential foreign key, you can do:

``` python
class ResUsers(BaseModel):
    create_uid = peewee.ForeignKeyField('self', related_name='users_created')
```

Documentation:
- http://docs.peewee-orm.com/en/latest/peewee/models.html#self-referential-foreign-keys
- http://docs.peewee-orm.com/en/latest/peewee/api.html#ForeignKeyField

Please let me know if this does not address your question.


Looks like I've understood difference between django ORM and Peewee in this part. 
Will update ticket, if something will go wrong. Thanks.