akumoth/peewee-issues huggingface dataset:
Issue: if a table join another table twice, the query cannot be done.
Author: zffl
State: closed
Created: 2012-08-05 07:48:52

Code as follow:

``` python

class A(db.Model):
    f1 = CharField()
    f2 = CharField()
    f3 = CharField()

class B(db.Model):
    f1 = ForeignKeyField(A)
    f2 = ForeignKeyField(A)
    f3 = CharField()
q = {A:['*'],
             B:['f1'],
             B:['f1'],
             }
SelectQuery(B,q).join(A, on='f1').switch(B).join(A, on='f2')

```

I also try alias , not work either.


This issue goes more-or-less to some of the core assumptions I made -- many things are simpler if multiple joins to the same table aren't allowed -- for example:
- in a complex query, the columns you select are represented as a dictionary keyed by model.  if a model appears twice (i.e. is joined on twice) how do you represent which to select from?
- when multiple models are joined on how do you express which you are querying in the where clause?

I don't think I'll be able to fix this anytime soon, in other words.  Sorry!  Looks like sqa can do this but is kinda hacky: http://stackoverflow.com/questions/5780516/how-to-join-detail-twice-on-a-master-in-sqlalchemy


One year on, it this still the case? I thought the Alias capability which allows for self joins would cause a new table alias to be created, but it doesn't appear to be the case.


I see now that it works if you use the aliased model in a join, but it can't be used as the root of a select query.


Correct -- you should be able to construct joins with aliased models, but not actually select them.  OK to close this out?
Issue: Support multiple annotation using different aggregates
Author: ternus
State: closed
Created: 2012-07-13 23:33:13

Using .annotate() multiple times on the same query fails:

``` python
annotated = Blog.select().annotate(Entry).annotate(Entry, Max('pub_date', 'max_pub'))
```

generates the SQL:

``` sql
SELECT t1."id", t1."title", MAX(t2."pub_date") AS max_pub FROM "blog" AS t1 INNER JOIN "entry" AS t2 ON t1."id" = t2."blog_id" GROUP BY t1."id", t1."title", *
```

which is a syntax error (stray \* near the end) in addition to being wrong.  The SQL generated should be something like:

``` sql
SELECT t1."id", t1."title", COUNT(t2.`pk`) AS count, MAX(t2."pub_date") AS max_pub FROM "blog" AS t1 INNER JOIN "entry" AS t2 ON t1."id" = t2."blog_id" GROUP BY t1."id", t1."title"
```

I've added a test case and a fix.


Hi @ternus sorry I missed your messages in IRC.  Will look into this today, thnx for the excellent report :)
Issue: Escaping mysql reserved words when creating foreign keys
Author: jonatanfan
State: closed
Created: 2012-07-03 15:51:16

Hi

Seems like reserved mysql words are not escaped when creating references for a table. Kind of a problem when a column is named `order`

Like so:
OrderProduct.create_table()
CREATE TABLE `orderproduct` (`id` INTEGER AUTO_INCREMENT NOT NULL PRIMARY KEY, `product_id` INTEGER NOT NULL REFERENCES product (id), `order_id` INTEGER NOT NULL REFERENCES order (id));


dupe of #98
Issue: Escaping mysql reserved words when creating foreign keys
Author: jonatanfan
State: closed
Created: 2012-07-03 15:51:16

Hi

Seems like reserved mysql words are not escaped when creating references for a table. Kind of a problem when a column is named `order`

Like so:
OrderProduct.create_table()
CREATE TABLE `orderproduct` (`id` INTEGER AUTO_INCREMENT NOT NULL PRIMARY KEY, `product_id` INTEGER NOT NULL REFERENCES product (id), `order_id` INTEGER NOT NULL REFERENCES order (id));
Issue: Database.executemany is not available
Author: karlb
State: closed
Created: 2012-06-27 14:32:33

In some cases I like to work directly with SQL by using Database.execute. However, I don't see a way to do what the DB-API executemany method does. In this specific case I'm trying to do multiple inserts this way.


I can call db.get_cursor().executemany, of course. It just expected executemany to be usable whereever execute can be used. So this is not important at all.


Yeah, I have not implemented executemany or executescript.  Many of the db-api 2.0 methods, while used internally, are not exposed -- this is intentional.  If you do need them, you can do like you've suggested, get a cursor.
Issue: Database Support
Author: cevarief
State: closed
Created: 2012-06-25 13:32:59

What is next database that will be supported by peewe? Any plan for firebird? 
For prototyping sqlite is enough, but for portable database in production embedded firebird could give more power...


I don't know much about firebird -- will have to look into it.  Currently, peewee's internal db calls rely on the driver supporting the python db-api 2.0.  It looks like kinterbasdb supports this, so integration should be really easy.  I was thinking personally that the next driver i'd like to look at is [apsw](http://apidoc.apsw.googlecode.com/hg/index.html), which provides much richer support for sqlite than the sqlite3 driver (virtual tables, etc).  If you want to see firebird, i'd suggest taking a crack at it yourself, the implementation should be simple - take a look at the code.

You will need to implement at the very least:
- an adapter class, which handles connecting to the database (here is [mysql](https://github.com/coleifer/peewee/blob/master/peewee.py#L224))
- a database class that uses the given adapter (here is [mysql](https://github.com/coleifer/peewee/blob/master/peewee.py#L537))

It is unlikely i'll be adding firebird support any time soon without learning a bit more about it and its advantages over, say, postgres mysql or sqlite which seem to run the gamut from prototype to large-scale production, but if you're inclined i'd be happy to merge in your changes if you write an implementation.


I've written some docs on writing a database backend:

http://peewee.readthedocs.org/en/latest/peewee/database.html#writing-a-database-driver
Issue: choices just like django or mongoengine
Author: ddorian
State: closed
Created: 2012-06-07 23:17:47

Add choices to fields like django and mongoengine.


i'm on the fence about this, should they be implemented as a check constraint or solely in python?  if you assign something that is an invalid choice, should it error out?  if so, what type of error?


i checked django, it seems you can assign invalid values to a field that specifies `choices`.  you can also save these invalid choices.  probably just for modelform validation.  so, options:
- db-level check constraint -- bad because if you want to change your choices that means schema change
- python-level on save
- python-level on assignment
- none at all, just for metadata and other libraries (flask-peewee, wtf-peewee)

I'm leaning towards none at all, same as django.


Maybe it should output a warning in DEBUG mode


I've added this in commit 2d076f069baf715d6dfd0070df861f0b3f417724

I've also added integration in the wtf-peewee bindings (wtforms form library integration)
https://github.com/coleifer/wtf-peewee/commits/master

I've also added a small example to the flask-peewee app:
https://github.com/coleifer/flask-peewee/commit/225f85d68fe39c0271b7a1a00b55ede1ede7f41f

Finally, there's no debug mode per se with peewee, so there's no warning
Issue: peewee 0.9.6 docs show transaction context manager, which isn't in 0.9.6
Author: rouge8
State: closed
Created: 2012-06-03 16:52:20

The [docs for 0.9.6](http://peewee.readthedocs.org/en/latest/peewee/cookbook.html?highlight=transaction#context-manager) show the context manager for transactions, but after installing 0.9.6 this wasn't supported.

Installing from `git:master` fixed it. Maybe time for 0.9.7?


thanks for catching this, i pushed a new version 0.9.7 and a new git tag.
Issue: Added auto_round and always_float attributes to DecimalField
Author: applegrew
State: closed
Created: 2012-06-02 20:59:55

- auto_round makes sure that if the value has higher precision than the
  number of decimal_places then it round it to that value before send
  that to db. This will save from some unexpected "Data truncation"
  warnings from MySQL (http://bit.ly/bWr1mn).
- always_float makes sure that in Python code always gets the value as
  float instead of Decimal. This way the return value can be easily
  intermixed with other floats. Also float can be faster than Decimal.


Not sure on the application of the float option, since there's a floatfield already.  As far as rounding, I suppose that would be handled by the DB when you specify a precision on the column, but this might need to be looked into a little more.


- Yes there is `FloatField` but it does not allow us to define number of decimals I need, (e.g. only two for currencies). Because of dynamic nature of Python it is difficult to work with codes where there is a possibility of mixing `float` data with the `Decimal` ones. So, it is important to normalize them to one type - maybe float. Well now I agree that in such a scenario it would indicate a design issue or there should be central point where the code should convert the `Decimal` returned to `float`, but then again I don't see anyway to implement custom getters and setters in Peewee `Model` to do it centrally. I will leave this open to your good judgement.
- DBs are responsible to accurately store the data, so if any data passed to it needs to be rounded which will result in loss of precision then it is going to throw a warning. This can happen when our `Decimal` has more decimals, or if we pass a `float`. Storing `23.221` in a `Decimal(5,2)` column in MySql throws 'Error 1265: Data truncation' warning, and I don't see any way to turn this off in DB. Even this warning is treated as error in Peewee, disrupting normal flow. By specifying `decimal_places` user has already consented with fact that he wants only that many decimals. Peewee already has enough information to lend user this helping hand. I really do not like warnings in my code. Now it can be argued that some users might want to warned about this loss. That is why I have added the `auto_round` flag to let user turn this behavior on or off. In fact it is off by default so that Peewee's current behavior do not change.


Added in 4c3af4ccc83dad38a369a1b8e3121de1d6838a7b -- thanks for the help!
Issue: The same record different instances save() behavior
Author: coderbuzz
State: closed
Created: 2012-05-25 16:46:51

**Please considering a bug:**
I'm in situation which fetching the same db record but different instance, each modified somewhere.

``` python
    item1 = get_item(filter='A') # return new instance of rec A
    item2 = get_item(filter='A') # also return new instance of rec A

    # item1 and item2 refer to the same db record, but different instance

    item1.some_field = 'some value'
    item1.save()

    item2.other_field = 'other value'
    item2.save()
```

The **item1** data won't saved correctly after committing transaction (different field changes)


This is correct, calling a model's save method will wholesale update the row in the database with the field data present on the model instance **at the time of saving**.  item2 will not automatically receive changes made to item1, since they are separate instances.  This is intended behavior.

If you want to only update a single column, use `update`:

``` python

ItemModel.update(other_field=item2.other_field).where(id=item2.id).execute()
```
Issue: NULL value for BooleanField returns False instead of None
Author: maiksprenger
State: closed
Created: 2012-05-23 09:46:09

I have a User object that can has, among others, the following fields.

```
class User(BaseModel):
    class Meta:
        db_table = 'users'

    user_id = peewee.PrimaryKeyField()
    crm_id = peewee.CharField(null=True)
    username = peewee.CharField()
    password = peewee.CharField()

    lost_pw_key = peewee.CharField(null=True)
    lost_pw_key_creation = UnixtimeField(null=True)

    lat = peewee.FloatField(null=True)
    lat = peewee.FloatField(null=True)

    birthday = UnixdateField(null=True)
    responds_quickly = peewee.BooleanField(null=True)
```

I use a MySQL database. The responds_quickly field is a BOOLEAN that allows NULL values. The lost_pw_key is a VARCHAR that allows NULL.
I have created a minimal user that only has required fields set. So, both responds_quickly and lost_pw_key are NULL in the database. But whereas the CharField is evaluated to be None, the BooleanField evaluates to False, which for me is unexpected (and wrong) behavior. There is alos FloatFields and IntegerFields in the model, and all of them evaluate to None as well.

```
> u.responds_quickly
False

> u.responds_quickly is None
False

> u.lost_pw_key is None
True
```

If you do not consider this a bug, how can I differentiate between a 0 and a NULL value for BooleanFields? I gladly assist if you need further info.
Issue: Support for model1 != model2
Author: wtld
State: closed
Created: 2012-05-23 09:42:59

``` python
>>> from models import Test
>>> t1 = Test.get(id=1)
>>> t2 = Test.get(id=1)
>>> t1 == t2
True
>>> t1 != t2
True
```

I think that `__ne__` method as opposite to `__eq__` should be added to the Model class.
Issue: Add schema support for Postgresql
Author: redapple
State: closed
Created: 2012-05-21 17:33:17

Hi,

I've been playing a bit with peewee and had to add a few things to inspect a Postgresql database within a schema.
I haven't done much testing apart from this local Postgresql (a MusicBrainz DB mirror).

Any thoughts?

Cheers.
Paul.


I ran w/some of your ideas and have added functionality for switching search path and specifying schema when introspecting:

bc465f467d0f3ccbf03c1b49ea2b2ecb04c0c765
963cfe41c9af773be934e36fb5e3a17eaeb85ff6


Cool.
I'll continue learning peewee.
Cheers,
Paul.
Issue: add, create, remove, clear from ReverseForeignRelatedObject
Author: maxekman
State: closed
Created: 2012-05-14 11:54:55

Have you thought of adding some helpers to (I think) ReverseForeignRelatedObject, similar to what Django have?

Here are their docs describing it, a little bit down on the page: https://docs.djangoproject.com/en/dev/topics/db/queries/#following-relationships-backward

Sorry for spamming with ideas, I just think peewee is worth it!


First off, thanks for the suggestions -- I appreciate your interest in the project!  This is a pretty nice API that django has, but i don't believe its something I'll add.  The 'reverse relation' is exposed as a simple select query, and I think that is a good fit.  The "automatic" filter is there on the select queyr, but i don't like adding an automatic attribute on creation or deletion.  The code is not really much shorter, either:

``` python

# django-like api
blog.entries.add(title='awesome post', content='lorem ipsum')

# current api
Entry.create(blog=blog, title='awesome post', content='lorem ipsum')
```

For now, i'm going to pass on this one.


Fair enough, I was not sure about it either, just read about it in the Django docs. Pretty cool though that the Django docs works somewhat for your project as well.
Issue: Hooks for CRUD
Author: maxekman
State: closed
Created: 2012-05-11 23:24:02

I'm trying to add this minimalistic FSM (https://github.com/oxplot/fysom) to a peewee model and just storing the state as a char field. The problem I'm having is restoring the state when selecting or creating models from elsewhere. Saving a state change is easy using the FSM callbacks. I have tried using a overloaded init, but that is called before the fields are pulled from the db. I have also tried overloaded class methods for create and and get, but that doesn't cover all cases (the filter method seems tricky to hack into from a subclass). Lastly, all my tries so far results in very un-pythonic code...

Have you though about adding hooks like the Django signals to perform custom pre/post CRUD? Some very simple callbacks would be enough to do some post init of more complex in-memory data structures (like a FSM) that depends on fields in the model.

Django uses these (https://docs.djangoproject.com/en/dev/ref/signals/):

``` python
pre_init
post_init
pre_save
post_save
pre_delete
post_delete
```

I think these hooks adds just enough to be a powerful tool without making peewee less minimalistic.

What do you think?


I'm on the fence about this...feel like the signals should be able to exist at the application level as opposed to the database level.


Sure, as I see it they should be on each model, that's how I was imagining to use them.

I my FSM case I would load the local in-memory FSM object after the model is loaded from the db, and I would imagine doing it on a post_init method of my actual model that have the FSM field.

A naming convention would probably be the best option, for example having the above methods as abstract methods on peewee.Model and just override any of them you wish to use in your own models.

Another option is setting each callback you want to use on the class level in your model.


Since save() and delete_instance() can easily be overridden and will do the right thing, I'll leave them as-is.  I have added a hook on the model instance named `prepared` which will be called when the row is ready and has received data from the db cursor.  You can now write:

``` python

class FSM(Model):
    state = CharField()

    def prepared(self):
        self.do_something_with_state()
```

This will be called when a model is populated from the db, whether called by
- select
- filter
- get

It will not be called when "related" models are selected, though, for instance

``` python

for entry in Entry.select({Entry: '*', Blog: '*'}):
    # entry will be prepared, but entry.blog will not
    pass
```


Cool, will try it out!
Issue: Database Creation
Author: maxekman
State: closed
Created: 2012-05-05 10:36:32

I need to read in the db config from a file which currently does not work with the way the Meta class is used. It creates a default "peewee.db" if the database attribute is not set when parsing the model classes. Could this be delayed until a Model is actually used?

I just read this thread on the mailing list about the same issue and wondered if this patch (or similar) could be applied to the main branch?

http://groups.google.com/group/peewee-orm/browse_thread/thread/53a50c4bb63e282b

Thanks,
Max


I've been thinking about how to fix this... I'm going to assume that a model is "actually used" whenever the db connection is either explicitly or implicitly opened.  If that's the case, then it really isn't a problem to structure your code such that you modify the underlying db connection on-the-fly.  The most important thing is that you don't want a bunch of `Database` instances running around, since each will open its own connection to the server -- in other words, there should be only one `Database` for each database you intend to connect to.

What I think I'll do is allow a user to set a db to "deferred" when initializing, then if you attempt to connect to the db (either by explicitly calling connect() or by trying to issue a query) it will raise an exception if the db hasn't been properly initialized.


If readthedocs ever updates, you can find docs here http://readthedocs.org/docs/peewee/en/latest/peewee/cookbook.html#deferring-initialization -- the basic idea is passing `None` as the db name will mark the db as deferred and an exception will be raised if you try and use it.  You can then properly init the db by calling `db.init()`:

``` python

>>> db = SqliteDatabase(None)
>>> db.deferred
True

>>> db.connect()
Exception raised

>>> db.init(':memory:')
>>> db.connect()
```


Very nice, I'll give this a try later. Is it published to pypi yet?


Just pushed a new version, 0.9.6 -- let me know how it works out!


This works perfectly!

The new best practice is probably to use a uninitialized db in the base model:

``` python
class BaseModel(peewee.Model):
    class Meta:
        database = peewee.SqliteDatabase(None)
```

And later init it:

``` python
BaseModel.Meta.database.init('db.sqlite')
```
Issue: Peewee ignores given database name. Puts peewee.db instead.
Author: DDevine
State: closed
Created: 2012-04-26 14:16:00

I had a database called database.db, then I opened a python prompt and did the following:

> > > import peewee
> > > from blog import Entry
> > > db = peewee.SqliteDatabase('database.db')
> > > db.connect()
> > > Entry.create_table()
> > > entry = Entry()
> > > entry.title = "test"
> > > entry.tags = "derp blah"
> > > entry.save()

This created peewee.db and created the table inside it, while my database.db remained empty. If I do the same but with a completely new database file name (say, "test.db") then test.db is not created and when I try to do create_table() it gives me an error saying that the table already exists due to it actually using pewee.db intead.

I am using peewee-0.9.5 on Python 2.7.3


This sounds like a configuration issue.  Please refer to the docs here:

http://peewee.readthedocs.org/en/latest/peewee/cookbook.html#creating-a-database-connection-and-tables

``` python

db = SqliteDatabase('database.db')

class Entry(Model):
    # foo bar baz

    class Meta:
        database = db # <--- I bet you're missing this
```


I bet you're right. Sorry!
Issue: on windows, peewee-0.9.4 line 1965, SyntaxError: invalid syntax
Author: huaitwooos
State: closed
Created: 2012-04-23 17:59:08

C:\Download\dev\flask-peewee\coleifer-peewee-6fb13f1\example>python
Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.

``` python
>>> import peewee
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\python25\lib\site-packages\peewee-0.9.4-py2.5.egg\peewee.py", line 19
65
    return datetime.datetime(*time.strptime(value, '%Y-%m-%d %H:%M:%S')[:6], mic
rosecond=usec)

       ^
SyntaxError: invalid syntax
```


updated formatting


I'm not quite clear on what is throwing that error, can you provide a little more information?


if i delete[ , microsecond=usec], the error will goto line 1994.

<pre>
Traceback (most recent call last):
  File "C:\Download\dev\peewee-0.9.4\a.py", line 1, in <module>
    import peewee
  File "C:\Download\dev\peewee-0.9.4\peewee.py", line 1994
    return datetime.time(*time.strptime(value, '%H:%M:%S')[3:6], microsecond=usec)
                                                                           ^
SyntaxError: invalid syntax
</pre>


So it sounds like windows python2.5 doesn't like microsecond?  I did a quick google and didn't find anything that jumped out at me.


JUST has  ERROR  ON PYTHON2.5.4. it's OK on Python2.7+windows.

I found two way to solve the problem on python 2.5.4:
- put microsencond keyword before the time.time.strptime(...):
  <pre>
  return datetime.datetime(microsecond=usec,*time.strptime('2011-2-3 12:23:34', '%Y-%m-%d %H:%M:%S')[:6])
  </pre>
- use a dict ref:
  <pre>
  return datetime.datetime(*_time.strptime('2011-2-3 12:23:34', '%Y-%m-%d %H:%M:%S')[:6], *_*{'microsecond':usec})
  </pre>


Fixed in 41de298683da7397b875597573a22a6683e19a6e -- thanks for the suggestions and for reporting the issue