akumoth/peewee-issues huggingface dataset:
Issue: Parenthesis for CompoundQueries in FROM clause
Author: demongoo
State: closed
Created: 2014-11-01 16:36:35

I made union query using `|` and was hoping to put it into `from_` of another query, but compiled query contains no parenthesis and results in error: `ProgrammingError at or near "SELECT`. Because inner query is not surrounded by parenthesis. Compiled output something like that:

```
inner = q1 | q2 | q3
q = Model.select(SQL('f1'), SQL('f2')).from(inner.alias('q')...

results in:

SELECT f1, f2 FROM q1 UNION q2 UNION q3 AS q
```

And fails. But if I try to do it manually like: `SELECT f1, f2 FROM (q1 UNION q2 UNION q3) AS q it is ok. (q1-q3 is just substitues for actual queries, to simplify example). Database is postgresql. Can we somehow fix it or am I missing some functionality for that?


What version of peewee are you using? In my testing locally it appears the parentheses are correct:

``` python
uq = User.select(User.username).where(User.username << ['a', 'b', 'd'])
        oq = (OrderedModel
              .select(OrderedModel.title)
              .where(OrderedModel.title << ['a', 'b'])
              .order_by())
        union = uq | oq
        print union.sql()
```

Prints:

``` sql
SELECT 1 FROM (
    SELECT "t2"."username" FROM "users" AS t2 WHERE ("t2"."username" IN (?, ?, ?)) 
    UNION 
    SELECT "t2"."title" FROM "orderedmodel" AS t2 WHERE ("t2"."title" IN (?, ?))
)
```


I use `peewee (2.4.0)`.
And just found that there's 2.4.1 available. Please, don't close the issue for a couple of days, ok? I'll check with 2.4.1 tomorrow and will write here.


Sure thing, I added a unittest which shows that the parentheses are present.


Indeed, seems it had been fixed recently. After upgrading it has parenthesis. Sorry for bothering. peewee is really cool piece of software. :)


No problem at all! Thanks for opening the issue, glad it's fixed for you.
Issue: Just and idea / discussion. Extensions support?
Author: rudyryk
State: closed
Created: 2014-10-30 21:05:50

Hello Charles!

Just an idea, what do you think of "standard" extensions support for peewee like in Flask? So everyone who wants to write extension for peewee creates package like "peewee_foobar" and peewee has special hook for importing these package with:

```
import peewee_ext.foobar
```

So we can put "stable" extensions to that namespace and provide a simple convention for creating extensions via simple package naming.

This is not about less verbosity. Just about a bit cleaner standard. I've played with it and it worked: https://github.com/rudyryk/peewee-ext


Technically all is necessary is to add some reference to docs and add "peewee_ext" package from that sample to distribution.


> Just an idea, what do you think of "standard" extensions support for peewee like in Flask?

I thought that Flask had moved away from that type of packaging.

Python packaging is already confusing enough, I say let's just have peewee plugins be normal python packages.


> I thought that Flask had moved away from that type of packaging.

They moved away from "namespace packages" and now Flask packages are normal packages, but there's a "import redirect" for "flask.ext.name" to "flask_name", so extension just define regular Python package "flask_name" and Flask is aware of the rest.

> Python packaging is already confusing enough, I say let's just have peewee plugins be normal python packages.

Well, I agree with that! :)

I just like that Flask extensions are well organised, so I've explored their approach.


This redirect rule was designed as a workaround for moving from old-style namespace packages to new style packages. So, it's more necessary step than initial intention. But ironically this workaround due to tight restriction has lead to common naming convention, which Django doesn't have for example :)


Interesting about the import redirection... At any rate, I think I prefer the default style, even at the loss of a little consistency in naming.
Issue: python_value not being called on create()
Author: alexlatchford
State: closed
Created: 2014-10-29 02:09:11

Hi Charles,

Just found another minor issue, this set of tests should demonstrate the problem.

```
import unittest
from datetime import datetime
from peewee import SqliteDatabase, Model, PrimaryKeyField, DateTimeField


db = SqliteDatabase(':memory:')


class MyModel(Model):
    id = PrimaryKeyField()
    my_field = DateTimeField()

    class Meta:
        database = db


class DatetimeConversionTest(unittest.TestCase):

    def setUp(self):
        db.connect()
        MyModel.create_table()

    def tearDown(self):
        MyModel.drop_table()
        db.close()

    def test_create(self):
        my_model = MyModel.create(my_field="2014-01-01 00:00:00")
        # Throws, actually returns "2014-01-01 00:00:00"
        self.assertEqual(my_model.my_field, datetime(2014, 1, 1, 0, 0, 0))

    def test_get(self):
        my_model = MyModel.create(my_field="2014-01-01 00:00:00")
        my_model2 = MyModel.get(MyModel.id == my_model.id)
        self.assertEqual(my_model2.my_field, datetime(2014, 1, 1, 0, 0, 0))  # Passes fine


if __name__ == "__main__":
    unittest.main()
```

I think it's just a case of running the python_value function over any arguments passed into the model. Not sure if this is something you this is something you'd consider a bug or not.

Cheers,
Alex


I don't think this is a bug, since you are creating the objects from Python, but I appreciate you opening the issue.


Yeah figured that might the response, I'll fix it on my end then. Cheers :)
Issue: Why does saving a model in a loop over a query go twice over the first two items?
Author: kramer65
State: closed
Created: 2014-10-28 16:46:54

I'm trying to save something for every user in a db, but to my surprise the loop goes double over the first two items. 

```
#!/usr/bin/env python
import os, random, string
from peewee import *

DB_NAME = 'people.db'
db = SqliteDatabase(DB_NAME)

class User(Model):
    name = CharField()
    class Meta:
        database = db

class PM(Model):
    user = ForeignKeyField(User)
    text = TextField()
    class Meta:
        database = db

try:
    os.remove(DB_NAME)
    print 'Deleted the existing DB'
except OSError:
    print 'No db present. Creating one.'

db.create_tables([User, PM])

for name in ['A', 'B', 'C']:
    u = User()
    u.name = name
    u.save()

print 'We\'ve got so many users: ', User.select().count()

for user in User.select():
    print 'We\'re now storing a pm for user: ', user.name
    pm = PM()
    pm.user = user
    pm.text = ''.join([random.choice(string.ascii_letters) for n in xrange(25)])
    pm.save()  # comment this line out and see what happens.
```

This prints out:

```
We've got so many users:  3
We're now storing a pm for user:  A
We're now storing a pm for user:  B
We're now storing a pm for user:  A
We're now storing a pm for user:  B
We're now storing a pm for user:  C
```

When you comment out the last line, it correctly prints out:

```
We've got so many users:  3
We're now storing a pm for user:  A
We're now storing a pm for user:  B
We're now storing a pm for user:  C
```

I tried whether this is caused by the ForeignKeyField, but if I do the same with an unrelated table ([code here](http://pastebin.com/ayWGbqkT)) I see the same behaviour.

Does anybody know what's wrong here?

ps. I have the feeling that I've encountered this before, but I can't remember how I dealt with it back then and I can't find any issues from myself about this.


Just to add to the confusion. If I run the loop one time before I run it while saving, it doesn't loop 3 times:

```
users = User.select()
for user in users: pass

for user in users:
    print 'We\'re now storing a pm for user: ', user.name
    pm = PM()
    pm.user = user
    pm.text = ''.join([random.choice(string.ascii_letters) for n in xrange(25)])
    pm.save()
```

This also correctly prints out:

```
We've got so many users:  3
We're now storing a pm for user:  A
We're now storing a pm for user:  B
We're now storing a pm for user:  C
```

but when I remove the line `for user in users: pass` it does this again:

```
We've got so many users:  3
We're now storing a pm for user:  A
We're now storing a pm for user:  B
We're now storing a pm for user:  A
We're now storing a pm for user:  B
We're now storing a pm for user:  C
```

Any idea why a loop before the second loop would influence the behaviour?


http://docs.peewee-orm.com/en/latest/peewee/database.html#common-pitfalls-with-sqlite
Issue: Fix typo in db_url.py
Author: malea
State: closed
Created: 2014-10-27 01:48:14

This was breaking on Heroku with the following error:

```
peewee.OperationalError: invalid connection option "post"
```

I checked the source code to see where this `"post"` was coming from, and here it was!


Wow, that's embarrassing! Thanks for the fix!
Issue: Add in detection of closed connections in the pool
Author: alexlatchford
State: closed
Created: 2014-10-08 03:27:15

Hi Charles,

I've got a little problem with connection pooling, it works great up until the point I have to kill a hanging postgres query or stop the database service. Any requests on that same connection get an `InterfaceError: connection already closed` from the psycopg2 library because it successfully retrieves a connection from `_connect`.

The problem is that the connection pool is only aware of connections killed by the application and not the database server itself. This pull is my attempt to monkey patch a solution for postgres, I had a brief look into the MySQLdb and pymysql docs to try to also get a good solution for that but fell short so thought I'd pass it over to you to see what you thought.

Other than this minor problem, (which can be fixed by restarting the webserver, thus flushing the connection pool), this module has been running really well for me in testing, thanks again :)

Cheers,
Alex


Since this is not implemented for MySQL I will pass on merging it as-is, but I'll think about how this might work as it seems like a valid issue.


Awesome, cheers Charles. I'll take another look into MySQL tomorrow to see what I can do from my end. Would be great to close this bug from my perspective :)


Did you have any thoughts about how to develop this? I'm willing to lend a hand but don't want to go in the wrong direction if this approach isn't what you're thinking.


I added a hook method which I believe should make it easy for you to implement your changes: ad50a51f9fa8669b8611964878d1ad0d87aa710e

Just subclass and override the `_is_closed()` method.


Awesome that works for me, one problem is that I need to use the `conn` itself within _is_closed, can you add that as a param to the definition please. I just checked and there is no sane way of going from the `id()` of an object to the object itself in python :)


Fixed in d442a92309c0ec437c18aa92293fe256933b4cf8
Issue: Cross database join
Author: RealJTG
State: closed
Created: 2014-10-06 10:13:43

Is it possible to perform cross database join somehow on the same MySQL instance? For example:

``` SQL
SELECT A.value, B.value
FROM db1.foo A 
JOIN db2.bar B ON A.id = B.id
```


Interesting. Peewee does not support this out of the box, so your best option right now is to probably use the `RawQuery` helper ([docs](http://docs.peewee-orm.com/en/latest/peewee/api.html#RawQuery)).

Another option is to take a look at how the `PostgresqlDatabase` supports a "schema" argument. The schema is only used to prefix table and column names, so maybe it could be repurposed to prefix with the database name instead?


`schema` seems to work fine. I'm using flask-peewee and my DB class now looks like this (with delayed initialization https://github.com/coleifer/flask-peewee/pull/154 and multiple configurations support)

``` python
class SchemaProxy(object):
    __slots__ = ['obj']

    def __init__(self):
        self.initialize(None)

    def initialize(self, obj):
        self.obj = obj

    def __str__(self):
        return self.obj or None


class ConfigurableDatabase(Database):

    def __init__(self, app=None, config_key='DATABASE'):
        self.config_key = config_key
        self.schema = SchemaProxy()
        super(ConfigurableDatabase, self).__init__(app)

    def load_database(self):
        self.database_config = dict(self.app.config[self.config_key])
        try:
            self.database_name = self.database_config.pop('name')
            self.database_engine = self.database_config.pop('engine')
            self.schema.initialize(self.database_name)
        except KeyError:
            raise ImproperlyConfigured('Please specify a "name" and "engine" for your database')

        try:
            self.database_class = load_class(self.database_engine)
            assert issubclass(self.database_class, PeeweeDatabase)
        except ImportError:
            raise ImproperlyConfigured('Unable to import: "%s"' % self.database_engine)
        except AttributeError:
            raise ImproperlyConfigured('Database engine not found: "%s"' % self.database_engine)
        except AssertionError:
            raise ImproperlyConfigured('Database engine not a subclass of peewee.Database: "%s"' % self.database_engine)

        self.database = self.database_class(self.database_name, **self.database_config)

    def get_model_class(self):
        class BaseModel(Model):
            class Meta:
                database = self.proxy
                schema = self.schema

        return BaseModel
```

With `werkzeug.local.LocalProxy` even cleaner, but not sure about side effects

``` python

class ConfigurableDatabase(Database):

    def get_model_class(self):
        class BaseModel(Model):
            class Meta:
                database = self.proxy
                schema = LocalProxy(lambda: self.database_name)

        return BaseModel
```


Neat. I'm going to close as it sounds like your immediate issue is resolved. Please comment if this is not the case.
Issue: What is the meaning of Model.dirty_fields?
Author: ak4nv
State: closed
Created: 2014-10-03 01:59:27

Just a code:

``` python
>>> from blueprints.auth.models import User
>>> user = User.get(User.id == 1)
>>> user.dirty_fields
[<peewee.PrimaryKeyField object at 0x7f8d2e948da0>, <peewee.CharField object at 0x7f8d2e9487b8>, <peewee.CharField object at 0x7f8d2e948a58>, <peewee.CharField object at 0x7f8d2e948898>, <peewee.CharField object at 0x7f8d2e948b00>, <peewee.CharField object at 0x7f8d2e948b38>, <peewee.CharField object at 0x7f8d2e948b70>, <peewee.ForeignKeyField object at 0x7f8d2e948ba8>, <peewee.DateTimeField object at 0x7f8d2e948be0>, <peewee.CharField object at 0x7f8d2e948c18>, <peewee.BooleanField object at 0x7f8d2e948c50>, <peewee.BooleanField object at 0x7f8d2e948c88>, <peewee.BooleanField object at 0x7f8d2e948cc0>]
>>> 
```

I expected to get an empty list. Am I wrong?


Looks like a bug -- the `dirty_fields` property should return an empty list for instances that are fetched from the database. Thanks for letting me know!


Yeah, I read it in docs.
I'll wait a bugfix. I need this functionality in my small project. :)
Issue: Order of join clause with alias join and select can lead to invalid configurations
Author: jakedt
State: closed
Created: 2014-09-24 21:04:31

I'm not sure how to describe the problem accurately, so here is a repro case:

``` python
from peewee import Model, SqliteDatabase, CharField, ForeignKeyField, create_model_tables

db = SqliteDatabase(':memory:')

class Person(Model):
    name = CharField(default='aperson')
    class Meta:
        database = db

class RelatesToPerson(Model):
    name = CharField(default='arelationship')
    person = ForeignKeyField(Person)
    class Meta:
        database = db

create_model_tables([Person, RelatesToPerson])

PersonAlias = Person.alias()

a_person = Person.create()
RelatesToPerson.create(person=a_person)

print str(RelatesToPerson
          .select(RelatesToPerson, PersonAlias)
          .join(PersonAlias, on=(PersonAlias.id == RelatesToPerson.person))
          .get().id) + " <- an instance!"

print (RelatesToPerson
       .select(RelatesToPerson, PersonAlias)
       .join(PersonAlias, on=(RelatesToPerson.person == PersonAlias.id))
       .get().id)
```

output

``` bash
$ python repro.py
<__main__.Person object at 0x10b201c50> <- an instance!
1
```


Thanks for sharing some code to reproduce this!


Basically what peewee is trying to figure out is how to patch the joined instance (`PersonAlias`, which is actually a `Person`) onto `RelatesToPerson` given the join predicate. There's some special-casing around aliases and expressions -- I think there are some undiscovered bugs in the relevant block of code. I'm working on a complete refactor to hopefully clean this up and make the behavior more predictable.


I've got things partially fixed in 55373892a7454052c54a5b8bc9ba7d9da72aeeb2


I believe this is now fixed.
Issue: Query iteration bug?
Author: frdfsnlght
State: closed
Created: 2014-09-19 18:57:40

``` python
from peewee import *
from datetime import datetime

db = SqliteDatabase('test.db')

class A(Model):
    id = PrimaryKeyField()
    date = DateTimeField()
    class Meta:
        database = db

A.create_table(fail_silently = True);
if not A.select().count():
    A(date = datetime.now()).save()
    A(date = datetime.now()).save()

print('%d records' % A.select().count())
count = 0;
for a in A.select():
    count += 1
    print('%d: %d' % (count, a.id))
    a.save()
```

Output:

```
>python test.py
2 records
1: 1
2: 2
3: 1
4: 2
```

Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)] on win32

If I comment the line "a.save()", it iterates only twice, as I would expect. Am I doing something wrong?


Check out:
- https://github.com/coleifer/peewee/issues/12#issuecomment-5614404
- https://github.com/coleifer/peewee/issues/81
Issue: Query bug
Author: ak4nv
State: closed
Created: 2014-09-19 07:36:16

``` python
    tasks = Task.select()\
      .where(Task.deadline.year == year)\
      .where(Task.performer << performers)\
      .where(Task.state << (States.ACCEPTED, States.CLOSED))
```

peewee.OperationalError: user-defined function raised exception

``` python
    tasks = Task.select()\
      .where(Task.performer << performers)\
      .where(Task.state << (States.ACCEPTED, States.CLOSED))\
      .where(Task.deadline.year == year)
```

works fine

Also

``` python
    tasks = Task.select().where(
      (Task.performer << performers) and
      (Task.state << (States.ACCEPTED, States.CLOSED)) and
      (Task.deadline == year)
    )
    print(tasks)
```

result

```
<class 'tm.tasks.models.Task'> SELECT "t1"."id", "t1"."title", "t1"."task_type", "t1"."desc", "t1"."deadline", "t1"."customer_id", "t1"."performer_id", "t1"."related_proj_id", "t1"."related_task_id", "t1"."state" FROM "tasks" AS t1 WHERE ("t1"."deadline" = ?) ORDER BY "t1"."id" DESC [2014]
```

and

``` python
    tasks = Task.select().where(
      (Task.performer << performers) &
      (Task.state << (States.ACCEPTED, States.CLOSED)) &
      (Task.deadline == year)
    )
    print(tasks)
```

result

```
<class 'tm.tasks.models.Task'> SELECT "t1"."id", "t1"."title", "t1"."task_type", "t1"."desc", "t1"."deadline", "t1"."customer_id", "t1"."performer_id", "t1"."related_proj_id", "t1"."related_task_id", "t1"."state" FROM "tasks" AS t1 WHERE ((("t1"."performer_id" IN (SELECT "t2"."id" FROM "users" AS t2 WHERE ("t2"."group_id" = ?))) AND ("t1"."state" IN (?, ?))) AND ("t1"."deadline" = ?)) ORDER BY "t1"."id" DESC [1, 'accepted', 'closed', 2014]
```

Python 3.4
Flask 0.10.1
peewee 2.3.2
psycopg2 2.5.4


First of all, your use of the python `and` keyword is incorrect -- see [query operators table](http://docs.peewee-orm.com/en/latest/peewee/querying.html#query-operators), particularly the section on logical operators.

As to the other examples, I will take a look. Your final query (using `&`) looks like it should work but I'm surprised the chained calls to .where do not since those should be equivalent.


> > peewee.OperationalError: user-defined function raised exception

Is it possible that in the first query you have a `None` value in the `deadline` column? I wonder if SQLite is running that filter first in the query that's failing, and it succeeds in the other examples because the result-set has already been filtered.


> Is it possible that in the first query you have a None value in the deadline column? 

Bingo!
My apologies. :)


Fixed in ffbfb9c024d5a63e895888b7f6eee46ad7f39b4b
Issue: Integration with mongodb?
Author: kramer65
State: closed
Created: 2014-09-18 11:59:29

I'm using Peewee in more and more projects now, but I'm currently working on a project in which we need to use MongoDB. Is there a way that I can make Peewee work with MongoDB? All tips are welcome!


I think you'll find that its better to use an existing library. Or be prepared to do a really close reading of the source code!


Basically you're going to be dealing with replacing the database classes and the query compiler. The model and field semantics are probably different as well. Please comment if you'd like to discuss more, but frankly I think you're barking up the wrong tree.
Issue: Using Peewee with Twisted
Author: fikisipi
State: closed
Created: 2014-09-18 10:27:06

At the moment, there is no way to use peewee with async Twisted.

Twisted handles database connections using [ConnectionPool](https://twistedmatrix.com/documents/14.0.0/api/twisted.enterprise.adbapi.ConnectionPool.html)

```
from twisted.enterprise import adbapi
db= adbapi.ConnectionPool("MySQLdb", 
    host = "",
    user = "",
    passwd = "",
    db = "")

q = db.runQuery("CREATE TABLE test(msg VARCHAR(100))")
q.addCallBack(asyncCallBackFunctionHere)
```

Would it be possible to have support for ConnectionPool? It has about 4 methods that are used to execute and return data.


I have no plans to support twisted but I'd accept s pull request for a playhouse module. If you're interested, I'd start here: 

http://docs.peewee-orm.com/en/latest/peewee/database.html#adding-a-new-database-driver
Issue: hstore extension is required to initialise a postgresql database
Author: alexlatchford
State: closed
Created: 2014-09-16 23:35:13

I've been meaning to write this one up for a while.. I've just installed the extension every time to avoid this one :)

```
Traceback (most recent call last):
  File "peewee_bug.py", line 22, in <module>
    MyModel.create_table()
  File "/Users/alexlatchford/Code/peewee/peewee.py", line 3423, in create_table
    db.create_table(cls)
  File "/Users/alexlatchford/Code/peewee/peewee.py", line 2788, in create_table
    return self.execute_sql(*qc.create_table(model_class, safe))
  File "/Users/alexlatchford/Code/peewee/playhouse/postgres_ext.py", line 288, in execute_sql
    self.commit()
  File "/Users/alexlatchford/Code/peewee/peewee.py", line 2600, in __exit__
    reraise(new_type, new_type(*exc_value.args), traceback)
  File "/Users/alexlatchford/Code/peewee/playhouse/postgres_ext.py", line 279, in execute_sql
    cursor = self.get_cursor()
  File "/Users/alexlatchford/Code/peewee/playhouse/postgres_ext.py", line 266, in get_cursor
    return self.get_conn().cursor(name=name)
  File "/Users/alexlatchford/Code/peewee/peewee.py", line 2680, in get_conn
    self.connect()
  File "/Users/alexlatchford/Code/peewee/peewee.py", line 2667, in connect
    self.__local.closed = False
  File "/Users/alexlatchford/Code/peewee/peewee.py", line 2600, in __exit__
    reraise(new_type, new_type(*exc_value.args), traceback)
  File "/Users/alexlatchford/Code/peewee/peewee.py", line 2666, in connect
    **self.connect_kwargs)
  File "/Users/alexlatchford/Code/peewee/playhouse/postgres_ext.py", line 293, in _connect
    register_hstore(conn, globally=True)
  File "/Library/Python/2.7/site-packages/psycopg2/extras.py", line 775, in register_hstore
    "hstore type not found in the database. "
peewee.ProgrammingError: hstore type not found in the database. please install it from your 'contrib/hstore.sql' file
```


I'm not really sure what to do here -- it's kind of on the database user to install the extension if they want to use it.


The point is I don't want to use it, it's forcing me to install it to use peewee.


Oh, I see.


I've added a `register_hstore` parameter to `PostgresqlExtDatabase`. The default is `True` to avoid breaking backwards compatibility.