akumoth/peewee-issues huggingface dataset:
Issue: Foreign key to the composite primary key
Author: Maillol
State: closed
Created: 2015-11-01 19:38:36

Create unit test in order to reproduce bug when we create foreign key to the
composite primary key.


I've waffled about this some. I dislike implicitly creating the columns. Additionally, when querying, things seem like they might get a little weird, e.g.:

``` python
class Person(Model):
    first_name = CharField()
    last_name = CharField()
    class Meta:
        primary_key = CompositeKey('first_name', 'last_name')

# API from your PR:
class Pet(Model):
    owner = ForeignKeyField(Person)

# API I had considered:
class Pet(Model):
    owner_first_name = CharField()
    owner_last_name = CharField()
    owner = CompositeForeignKey(Person)
```

Each has its downsides. In the former, the related person's first and last name, which are present on the Pet, are abstracted away and not easily accessible. In the latter, I'm left wondering how useful that `owner` attribute is really going to be.


I agree with you, creating columns must be explicit.

currently the problem appear when we want create a weak entity  which contains an other weak entity. 
i.e:
Book -> Chapter -> Paragraph 
or
Train -> Wagon -> Seat

The PK Wagon is (PK train + num Wagon) and PK Seat is (PK train + num Wagon + num Seat).
The Wagon num field isn't unique, I can't create a foreign key on it, I must use composite FK. 

What do you think about API like this ?

``` python
class Pet(Model):
    owner_first_name = ForeignKey(Person, first_name)
    owner_last_name = ForeignKey(Person, last_name)

    class Meta:
        foreign_keys = [
            CompositeKey(owner_first_name, owner_last_name)
        ]
```


> What do you think about API like this ?

That would be confusing I think. By making `owner_first_name` and `owner_last_name` ForeignKeyFields, it is unclear what value you are supposed to assign to them. Typically with a foreign key field you would assign the related model instance (although it is possible to assign the underyling scalar key value). This "overloads" the concept of foreign keys in a way I find ambiguous.

If there were going to be an API, it would be this most likely:

``` python
class Pet(Model):
    owner_first_name = CharField()
    owner_last_name = CharField()
    owner = CompositeForeignKey(Person)
```

But as I said I have doubts about the utility of this.

I think that for now I am going to close this as _wontfix_, since there doesn't seem to me to be a clear path forward. You can specify table-level constraints as model meta options, so if you want schema-level referential integrity, you can do something like:

``` python
class Pet(Model):
    owner_first_name = CharField()
    owner_last_name = CharField()

    class Meta:
        constraints = [SQL('FOREIGN KEY(owner_first, owner_last) REFERENCES person(first, last)')]
```


Documentation: http://docs.peewee-orm.com/en/latest/peewee/models.html#indexes-and-constraints
Issue: Issue with `test_database()` helper
Author: coleifer
State: closed
Created: 2015-10-28 16:31:40

VIA: http://stackoverflow.com/a/33346767/254346

When using `test_database` I encountered problems with `test_db` not being initialized:

```
nose.proxy.Exception: Error, database not properly initialized before opening connection
-------------------- >> begin captured logging << --------------------
peewee: DEBUG: ('SELECT "t1"."id", "t1"."name", "t1"."count" FROM "counter" AS t1', [])
--------------------- >> end captured logging << ---------------------
```

I eventually fixed this by passing `create_tables=True` like so:

``` python
def test_timeline(self):
    with test_database(test_db, (User, Tweet), create_tables=True):
        # This data will be created in `test_db`
        self.create_test_data()
```

According to [the docs](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#test-utils) `create_tables` should default to `True` but it seems that isn't the case in the latest release of `peewee`.


The code indicates that `create_tables=True`:

``` python
class test_database(object):
    def __init__(self, db, models, create_tables=True, drop_tables=True,
                 fail_silently=False):
        self.db = db
        self.models = models
        self.create_tables = create_tables
        self.drop_tables = drop_tables
        self.fail_silently = fail_silently
```

This issue seems invalid.
Issue: Foreign key collision regarding related_name
Author: dzeban
State: closed
Created: 2015-10-28 15:50:52

I have 2 tables

``` sql
CREATE TABLE "Feed"
(
  id serial NOT NULL,
  url text,
  CONSTRAINT "Feed_pkey" PRIMARY KEY (id)
)
CREATE TABLE "Entry"
(
  id serial NOT NULL,
  link text,
  feed_id integer,
  CONSTRAINT "Entry_pkey" PRIMARY KEY (id),
  CONSTRAINT "Entry_feed_id_fkey" FOREIGN KEY (feed_id)
      REFERENCES "Feed" (id) MATCH SIMPLE
      ON UPDATE NO ACTION ON DELETE NO ACTION
)
```

Next I'm generating models with pwiz:

``` python
from peewee import *

database = PostgresqlDatabase('test', **{'host': 'localhost', 'user': 'ribbo'})

class UnknownField(object):
    pass

class BaseModel(Model):
    class Meta:
        database = database

class Feed(BaseModel):
    url = TextField(null=True)

    class Meta:
        db_table = 'Feed'

class Entry(BaseModel):
    feed = ForeignKeyField(db_column='feed_id', null=True, rel_model=Feed, to_field='id')
    link = TextField(null=True)

    class Meta:
        db_table = 'Entry'
```

But when I'm subclassing on Entry I get the error:

``` python
$ cat Entry.py 
from models import database
from models import Entry as EntryModel

class Entry(EntryModel):
    myattr = 'somestring'

    def some():
        print('Hi')

e = Entry()
```

```
$ python3 Entry.py 
Traceback (most recent call last):
  File "Entry.py", line 4, in <module>
    class Entry(EntryModel):
  File "/usr/lib/python3.4/site-packages/peewee.py", line 4224, in __new__
    field.add_to_class(cls, name)
  File "/usr/lib/python3.4/site-packages/peewee.py", line 1250, in add_to_class
    self.model_class._meta.name, self.name, self.related_name))
AttributeError: Foreign key: entry.feed related name "entry_set" collision with foreign key using same related_name.
```

The fun part is that if I change `class Entry(EntryModel)` class name to `class Entry2(EntryModel)` problem disappears.


I wouldn't consider this an error per se, just a quirk of the way peewee does backref validation. If you want to suppress the error, just declare your model with `validate_backrefs = False`:

``` python
class Entry(BaseModel):
    class Meta:
        validate_backrefs = False
```
Issue: Ability to verify inputs based on field type
Author: kostasdizas
State: closed
Created: 2015-10-26 21:30:16

Hello,

is it possible to verify whether a value is appropriate for a specific field? Is there a comparison/verification function? Is the equivalent primitive type stored somewhere in the field type (after a very quick search I didn't find one) in order to compare it with the type of the input value?
What's the best way to achieve that?


Peewee does not do any validation. The field's value is converted (if necessary) when the field is used in a query. When reading values from a cursor, the value is converted to Python when the model object is created.

I suppose you could add validation of some kind to your fields, but that would exist outside of peewee.
Issue: create_tables() fails
Author: timeyyy
State: closed
Created: 2015-10-24 09:48:51

on a sqlitedatabase, using create_tables() results in a 
peewee.OperationalError, table xyz already exists,

i am able to create the tables if i use create_tables()


You can pass `safe=True` and Peewee will not try to create it if it already exists.
Issue: peewee.IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (test_peewee_tree.node, CONSTRAINT node_ibfk_1 FOREIGN KEY (parent_id) REFERENCES node (id))') db = MySQLDatabase
Author: slukin1
State: closed
Created: 2015-10-23 17:18:00

peewee.IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (test_peewee_tree.node, CONSTRAINT node_ibfk_1 FOREIGN KEY (parent_id) REFERENCES node (id))')
db = MySQLDatabase


Uhh, did you make the `parent` foreign key nullable? This looks like a data integrity error rather than peewee.


Yes, parent foreign ey made nullable:

``` python
mysql_db = MySQLDatabase('test_peewee_tree')
class BaseModel(Model):
class Meta:
        database = mysql_db
class Node(BaseModel):
    label = CharField()
    parent = ForeignKeyField('self', default=0)
    left = IntegerField(default=0)
    right = IntegerField(default=1)
    depth = IntegerField(default=0)
def get_parent(self, level=1):
        parent = self
        for i in range(level):
            parent = parent.parent
        return parent
if __name__ == '__main__':

    Node.create_table()

    n1 = Node(label='root')
    n2 = Node(label='a')
    n3 = Node(label='b')
    n4 = Node(label='c')
    n1.append(n2)
    n1.append(n3)
    n2.append(n4)
```


Same peewee.IntegrityError: (1452....
if changed to
parent = ForeignKeyField('self', default=0, null = True)
or
parent = ForeignKeyField('self', db_column='parent_id', default=0, null = True)


No error peewee.IntegrityError: (1452...
with peewee version 2.2.4 and parent = ForeignKeyField('self', null = True)


With parent = ForeignKeyField('self', default=None, null=True) works well if peewee version 2.2.4
but with peewee version 2.6.4 error:
row = self.select().where(self.pk_expr()).first()
AttributeError: 'Node' object has no attribute 'pk_expr'


It's very difficult to read your comments as you have not bothered to format the code. I edited one of your comments so you can see how to do it.

For what it's worth, 2.2.4 is quite old (about a year and a half). Where is the `row = self.select().where(self.pk_expr())` coming from?


I use hierarchical models in a couple places as well as in the test suite. The right way to do it is

``` python
parent = ForeignKeyField('self', null=True)
```

If you aren't re-creating your tables every time you change the field then just changing the Python field deifnition won't alter the underlying schema which may have a not-null constraint.


For peewee version 2.6.4:
self.pk_expr() => self._pk_expr() # pk -> _pk
Issue: adding a .collate_nocase() query feature
Author: aerickson
State: closed
Created: 2015-10-21 21:06:31

Please let me know if you'd like me to make test and/or documentation changes.

Usage example:

```
order_by(Object.name).collate_nocase()
```


I guess this is unnecessary and only works on Sqlite.

fn.Lower() seems to work fine.

```
order_by(fn.Lower(Object.name))
```


Yeah you can also do:

.order_by(Clause(Model.field, SQL(' collate nocase')))
Issue: In meta ,how to set "order by synax"
Author: rfyiamcool
State: closed
Created: 2015-10-18 15:13:28

I'm use Peewee ORM.

I have a class such as this:

``` python
class Sample(PMBaseModel):
    id = peewee.PrimaryKeyField()
    location = peewee.CharField(max_length=255)
    position = peewee.IntegerField()

    class Meta:
        db_table = 'sample'
```

how i can use Meta Class for set default ordering and define Order type?


Well, I think setting a default ordering is _always_ a bad idea, but since you asked:

``` python
class Meta:
    order_by = ('location', '-position')
```

`order_by` is a tuple. The elements of the tuple are field names. If prefixed by a "-", then the field is ordered descending.


6969201d22676e4e7557949fcccd13d835a662a4 contains doc updates.
Issue: Exceeded maximum connections.
Author: Darnok99
State: closed
Created: 2015-10-16 00:09:58

We're using PooledPostgresqlExtDatabase in our application but it looks like it doesn't clean closed connections what leads to error from the subject. After digging a little in the PooledDatabase class i think the problem is when connection is stale. Then _connect invokes _close with parameter close_conn=True

``` python
if self.stale_timeout and self._is_stale(ts):
     logger.debug('Connection %s was stale, closing.', key)
     self._close(conn, True) # HERE
     ts = conn = None
```

and later in _close function in the same class we have 

``` python
if close_conn:
    self._closed.add(key)
    super(PooledDatabase, self)._close(conn)
```

which doesn't remove connection from _in_use and after some time this leads to "Exceeded maximum connections.". I guess changing close_conn from True to False in _connect function should do the trick.


`self._connections` and `self._in_use` should be mutually exclusive, though, right? i.e., the pool of connections we are pulling the connection /from/ should not be in use. The second branch there, the `elif key in self._in_use:` is there to pre-emptively remove stale connections on close, i.e. if the conn was very close to being stale when checked out, then while in use it became stale, we can simply close it when it's checked back in.

Are you sure you are not forgetting to close a connection somewhere in your app?


Hmm but I think second branch `elif key in self._in_use` won't be invoked since we have 

``` python
def _close(self, conn, close_conn=False):
        key = self.conn_key(conn)
        if close_conn:
            self._closed.add(key)
            super(PooledDatabase, self)._close(conn)

       # this one won't be invoked because close_conn is true
        elif key in self._in_use:
            ts = self._in_use[key]
            del self._in_use[key]
            if self.stale_timeout and self._is_stale(ts):
                logger.debug('Closing stale connection %s.', key)
                self._close(conn, close_conn=True)
            else:
                logger.debug('Returning %s to pool.', key)
                heapq.heappush(self._connections, (ts, conn))

```

Main thing is "where do we remove connection from _in_use collection" because check on this throws error I'm facing.


Can you share some of the code you're using to connect and close the conn? Or some more details about your app? The pool is covered by unit tests, so I'm not sure what I'm missing here.

My point is that I figured `self._connections` and `self._in_use` are mutually exclusive.


Hey, sorry for bugging you earlier. The problem is on our end, we removed stale time and still have same error. I can't provide you code samples because peewee is in quite many places in our project.
Sorry for shooting from the hip.
Cheers,
Konrad
Issue: MySQLDatabase missing __init__ function, cannot set commit_select when constructing connection
Author: pushinginertia
State: closed
Created: 2015-10-03 16:04:00

This is related to issue #431.

In order to use a cursor with MySQL and avoid an out of memory error, I need to use the `MySQLdb.cursors.SSCursor` class and disable commits on select.

It seems to me that defaulting to NOT committing on all select queries should be the default since select is a non-mutating operation, but it would be helpful to give the flexibility of overriding this value when instantiating the db connection.

In this specific case, by overriding the default cursor provided by the MySQL driver to `SSCursor`, I am changing the behaviour so that results are stored server-side and a call to `Database.execute_sql` attempts to execute a commit after initializing the cursor.

This results in the error: `Exception _mysql_exceptions.ProgrammingError: (2014, "Commands out of sync; you can't run this command now") in <bound method SSCursor.__del__ of <MySQLdb.cursors.SSCursor object at 0x10a706990>> ignored`

I can solve this problem by instantiating my connection as:

```
db = MySQLDatabase(
    'databasename',
    ...,
    cursorclass=cursors.SSCursor,
    commit_select = False,
)
```

However, `MySQLDatabase` does not have an `__init__` function and is defined as follows:

```
class MySQLDatabase(Database):
    commit_select = True
    .. more initialization of variables ..
```

Because of the above implementation, `commit_select` does not get set to `False` as I would expect. I suggest the following implementation instead:

```
class MySQLDatabase(Database):
    def __init__(commit_select=True, **kwargs):
        super(MySQLDatabase, self).__init__()
        self.commit_select = commit_select
        .. any other initialization ..
```


> It seems to me that defaulting to NOT committing on all select queries should be the default since select is a non-mutating operation

I pushed a commit to remove it in master, which will be in the next release. I know with postgres the reason I was doing that was to avoid connections hanging idle in transactions. I'm honestly not sure if mysql requires similar logic, but you can remove it for the time being by subclassing `MySQLDatabase`:

``` python
class MySQLDatabaseNoCommit(MySQLDatabase):
    commit_select = False
```


I'm going to revert this change as it has caused lots of confusion for MySQL users. Say what you will, but yeah, I think it's better for the users to not be confused. For the time being, you'll have to subclass MySQLDatabase and override the `commit_select` attribute:

``` python

class CustomMySQLDatabase(MySQLDatabase):
    commit_select = False
```
Issue: Confused about inconsistency of *DoesNotExist exceptions
Author: pitchforks
State: closed
Created: 2015-09-28 15:18:48

peewee-2.6.4

Suppose I have a model named `MyModel`.

I am trying to fetch a `my_model` row by using `MyModel.get(CONDITION)`. If such a row can't be found - an exception is raised. The displayed traceback says `peewee.MyModelDoesNotExist`.

If I try to catch this exception - the result does not change! Diving deeper with a debugger reveals that I should actually catch `MyModel.DoesNotExist`!

This makes the traceback displayed by peewee confusing and very misleading. Can you please change the traceback message to hint about the correct exception name that should be caught or is there any technical limitation for not doing so?


For anyone like me who stumbles upon this from google with the same question, just use the exception at the top level of the package, and ignore the concatenation of your model name onto the exception in the traceback. For example, for model `MyModel` with id field `id`:
```python
import peewee
from my_app.models import MyModel

def get_mymodel(requested_id):
    try:
        result = MyModel.get(MyModel.id == requested_id)
    except peewee.DoesNotExist:
        raise ValueError('No instance of MyModel exists at {}'.format(requested_id))
```
Issue: How do I avoid circular imports?
Author: danieldiekmeier
State: closed
Created: 2015-09-25 22:27:51

I use peewee in my webapp http://lesetagebu.ch, and it's great! Thank you. But I have a problem:

All my models lived in a models.py, but the file was several hundred lines long and hard to manage. So I refactored each model into its own file in ./models.

But now I have problems with circular imports. For example:

``` python
# models/user.py (irrelevant parts omitted)

from .rootmodel import RootModel
from .resolution import Resolution

class User(RootModel):
    username = CharField(unique=True)
    password = CharField()

    def resolution(self):
        year = arrow.now('Europe/Berlin').year

        try:
            resolution = self.resolutions.where(
                (Resolution.year == year)
            )

            if resolution.count() == 0:
                raise(DoesNotExist)
        except DoesNotExist:
            resolution = Resolution.create(
                amount=0,
                active=True,
                year=year
            )
        return sresolution
```

``` python
# models/resolution.py (irrelevant parts omitted)

from .rootmodel import RootModel
from .user import User

class Resolution(RootModel):
    amount = IntegerField()
    active = BooleanField()
    year = IntegerField()
    user = ForeignKeyField(User, related_name='resolutions')
```

This is the traceback:

```
Traceback (most recent call last):
  File "run.py", line 1, in <module>
    import lesetagebuch.main as main
  File "/Users/Daniel/dev/python/lesetagebuch/lesetagebuch/main.py", line 19, in <module>
    from lesetagebuch.models import *
  File "/Users/Daniel/dev/python/lesetagebuch/lesetagebuch/models/__init__.py", line 1, in <module>
    from .apikey import APIKey
  File "/Users/Daniel/dev/python/lesetagebuch/lesetagebuch/models/apikey.py", line 6, in <module>
    from .user import User
  File "/Users/Daniel/dev/python/lesetagebuch/lesetagebuch/models/user.py", line 14, in <module>
    from .resolution import Resolution
  File "/Users/Daniel/dev/python/lesetagebuch/lesetagebuch/models/resolution.py", line 8, in <module>
    from .user import User
ImportError: cannot import name 'User'
```

The problem is, I need the User in the resolution.py, because I need to define the Foreign Key, but I also need the Resolution in the user.py, because I need to be able to create a new Resolution instance.

In Django, there seems to be a way by using strings: http://stackoverflow.com/questions/4379042/django-circular-model-import-issue, but that doesn't work in peewee, because the first argument of ForeignKeyField needs to be a class.

Maybe the solution is quite obvious, but I can't find it. What do you recommend?


One possibility is, of course, to move the import of Resolution into the `resolution` function. This definitely works, but it feels dirty. :(


You can use `Proxy` for this.

http://docs.peewee-orm.com/en/latest/peewee/models.html?highlight=proxy#circular-foreign-key-dependencies
Issue: Make array field adapter independent from the connection.
Author: stas
State: closed
Created: 2015-09-24 17:55:59

Since the field does not depend on the database OIDs, calling `prepare()` is not necessary. This is also mentioned in the docs[1](http://initd.org/psycopg/docs/extensions.html#psycopg2.extensions.ISQLQuote.prepare). The reason this is not dependent on the connection since we don't try to adapt any special OIDs that can vary from one Postgres version to another.

We discovered this when a wrapped connection was used to adapt the new type.
Relevant: opbeat/opbeat_python#68

Thanks in advance for reviewing this.


Hmm, it looks like there's something strange with the way py3 handles the unicode string adapter.

This could be related to psycopg/psycopg2#331
I've tried a couple of things, including calling `adapted.prepare(conn)` and explicitly setting the connection encoding to `UTF8`, still on py3 I get the adapter result as bytea.

We can close this until the py3 support is not ready... :|
Issue: Wrong code sample in docs
Author: ael-code
State: closed
Created: 2015-09-23 17:11:02

I'm using peewee for the first time but it seems to me that the docs on [querying#joining-on-multiple-tables](http://peewee.readthedocs.org/en/latest/peewee/querying.html#joining-on-multiple-tables) has two wrong code samples:

``` python
User.join(Tweet).join(Comment)
```

``` python
# Join the Artist table on both `Ablum` and `Genre`.
Artist.join(Album).switch(Artist).join(Genre)
```

Is the `select()` missing?