akumoth/peewee-issues huggingface dataset:
Issue: Use of super on old style class `Model`
Author: soasme
State: closed
Created: 2014-01-20 02:16:22

I got an pyflake error on overriding `Model#save` method:

> [E1002, MyModel.save] Use of super on an old style class

Here is the example:

``` python
# -*- coding: utf-8 -*-

from peewee import Model
class MyModel(Model):

    def save(self, *args, **kw):
        return super(MyModel, self).save(*args, **kw)

a = MyModel()
print MyModel.mro()
# [<class '__main__.MyModel'>, <class 'peewee.Model'>, <class 'peewee.NewBase'>, <type 'object'>]
```

Is it possible to turn `Model` class to a `new-style` class?


I thought `Model` _was_ a new-style class?


Oops, it seems that pyflake does not have enough intelligence to recognize these styles:

``` python
def with_metaclass(meta, base=object):
    return meta("NewBase", (base,), {})

class BaseModel(type):
    def __new__(cls, name, bases, attrs):
        return super(BaseModel, cls).__new__(cls, name, bases, attrs)

class Model(with_metaclass(BaseModel)):
    def save(self):
        pass

class MyModel(Model):
    def save(self): # [E1002, MyModel.save] Use of super on an old style class
        super(MyModel, self).save()
```
Issue: postgresql: invalid last insert id when setting id manually
Author: appetito
State: closed
Created: 2014-01-16 16:14:38

```
In [17]: cat = Category.create(name='a')

In [18]: cat.id
Out[18]: 1L

In [19]: cat = Category.create(name='b')

In [21]: cat.id
Out[21]: 2L

In [22]: cat = Category.create(name='c', id=100)

In [23]: cat.id
Out[23]: 2L

=========

select * from categories;
 id  | parent_id | name 
-----+-----------+------
   1 |           | a
   2 |           | b
 100 |           | c
(3 rows)

```

may be we must use "INSERT **\* RETURNING id"?


> may be we must use "INSERT **\* RETURNING id"?

That is a postgres feature, unfortunately it won't work for Sqlite or MySQL... Setting an auto-incrementing ID manually is frowned on, but if you are bulk-loading data or for some reason need to control the ids, see these docs for some tips:

http://peewee.readthedocs.org/en/latest/peewee/cookbook.html#bulk-loading-data-or-manually-specifying-primary-keys
Issue: Dirty Flag
Author: soasme
State: closed
Created: 2014-01-16 03:43:45

Update all fields on saving will increase probability of `lost update`.
Such as: 

TransactionA do not update username field
TransactionB update username field
TransactionB save
TransactionA save. 

then we will lost username modification in TransactionB.
But if we just update necessary fields without username in TransactionA, it's okay.

we can easily call `model.save(only=model.dirty_fields)` to fix that.


Nice, thanks for this.
Issue: Support postgres column type 'json'
Author: extesy
State: closed
Created: 2014-01-14 17:01:38

Currently there is a problem with this column type because psycopg always unpacks json column data into a dict during load, and packs it during save (see _json.py file from psycopg2).

When peewee tries to interpret this dict as text, this causes python to serialize it, and this serialization makes it json-incompatible now so you can't save it back anymore.

Two possible alternatives are: either do what psycopg is doing and always treat this column as dict instead of string, or wrap it into json.dumps before exposing to the user and json.loads before passing to psycopg level. I'd personally prefer first option - seems cleaner.


@extesy -- peewee does not currently come with a JSON field type (if it did, it would be part of `playhouse.postgres_ext`).  It sounds like you may have started an implementation of one, otherwise I'm not sure how you have been using the field?  Additionally, are you registering a json adapter or using any psycopg2 APIs?

The more information you can provide the better, thanks!


The only thing I came up so far is to override default psycopg2's json handler by `psycopg2.extras.register_default_json(loads=lambda x: x)` but it looks like hack, not a normal approach to the problem. I've been using peewee only and this workaround is the only case where I use psycopg2 directly.

I think you are right and new JsonField needs to be added to playhouse.postgres_ext, similar to existing HStoreField, etc.


I've added a _very minimal_ Json field.  I plan on adding support for the `->` and `->>` operators, but since these are chainable I'd like to think about the implementation a bit.


633e5307d0fe19b3e299f2d6c5d7cd3ee1fd24ca
Issue: Foreign key attribute access in join
Author: bndr
State: closed
Created: 2014-01-12 20:45:24

Hi!

So I've run in to a bit of a problem.

I have models:

```
   class Trip(AbstractModel):

    date = DateTimeField()
    from_place = ForeignKeyField(Place, related_name="place_from")
    to_place = ForeignKeyField(Place, related_name="place_to")
```

```
   class Place(AbstractModel):

    name = CharField(unique=True)
    lat = FloatField()
    lon = FloatField()
    type_ = CharField()
```

And I need to calculate the great circle distance formula (Get the places around the from_place with predefined radius)

```
    rLat = math.radians(from_.lat)
    rLon = math.radians(from_.lon)
    earth_radius = 6371  # km
    # Great circle distance formula
    trips = Trip.select().join(Place).where(
        Trip.to_place == to_.id &
        Trip.date >= date &
        Trip.status == "active").where(
            (math.acos(
             math.cos(math.radians(Trip.from_place.lat)) * math.cos(
             rLat) * math.cos(math.radians(Trip.from_place.lon) - rLon)
             + math.sin(math.radians(Trip.from_place.lat)) * math.sin(rLat)) * earth_radius) < radius)
```

The problem is that I can't access Trip.place_from.lat attribute. I get an error: `AttributeError: 'ForeignKeyField' object has no attribute 'lat'`

Is there anyway to fix this/do this type of query? Or will I have to write this as raw SQL?


When you access the `lat` and `lon` attrs, you need to do so from the `Place` model.  You need to wrap your comparisons in parens because of operator precedence.  Lastly, the usage of the `math` module won't work at all the way you're trying to use it.  You will need to use database functions.

I did not test the code below, but hopefully this helps get you on the right track:

``` python
ToPlace = Place.alias()
FromPlace = Place.alias()
trips = (
    Trip
    .select()
    .join(ToPlace, on=(Trip.to_place == ToPlace.id))
    .switch(Trip)
    .join(FromPlace, on=(Trip.from_place == FromPlace.id))
    .where(
        (Trip.date >= date) &
        (Trip.status == 'active') &
        (fn.Acos(
            (fn.Cos(fn.Radians(FromPlace.lat)) * fn.Cos(rLat) * fn.Cos(fn.Radians(FromPlace.lon) - rLon)) +
            (fn.Sin(fn.Radians(FromPlace.lat)) * fn.Sin(rLat)) * earth_radius)) < radius))
```


Thanks! Your answer helped a lot! I finally got it working.

I'll just write some keywords here, so that people looking for the same thing, could find it: peewee great circle distance haversine formula, two gps points
Issue: where() calls are not chainable with update()
Author: grantcox
State: closed
Created: 2013-12-31 07:21:23

The documentation (http://peewee.readthedocs.org/en/latest/peewee/api.html#Query.where) states that "where() calls are chainable. Multiple calls will be “AND”-ed together.".  This works for SelectQuery:

```
Model.select().where(Model.id == 1).where(Model.status == 5)
SELECT ... FROM `model` AS t1 WHERE ((t1.`id`=1) AND (t1.`status`=5))
```

But does not work for UpdateQuery - only the final where() clause is included in the generated query.

```
Model.update(status=-1).where(Model.id == 1).where(Model.status == 5).execute()
UPDATE `model` SET `status`=-1 WHERE (`status`=5)
```

In this example the Model.id == 1 clause has been dropped, causing a dangerous update.

The clauses are all correctly included if they are specified in a single where():

```
Model.update(status=-1).where((Model.id == 1) & (Model.status == 5)).execute()
UPDATE `model` SET `status`=-1 WHERE ((`id`=1) AND (`status`=5))
```


Thank you so much for bringing this to my attention, you are absolutely right.
Issue: some error on foreign_key with Version 2.1.7 
Author: ghost
State: closed
Created: 2013-12-26 06:41:47

When I up to 2.1.7 today, tables con't create 。But in 2.16 I know it works good。

Below is what happened when I use it with Python3 and MySQL.

Traceback (most recent call last):
  File "E:/Work/Python/notepy/model/model.py", line 74, in <module>
    PostTag.create_table()
  File "C:\Python33\lib\site-packages\peewee.py", line 2683, in create_table
    db.create_table(cls)
  File "C:\Python33\lib\site-packages\peewee.py", line 2116, in create_table
    return self.execute_sql(qc.create_table(model_class, safe))
  File "C:\Python33\lib\site-packages\peewee.py", line 1229, in create_table
    return ' '.join(self.create_table_sql(model_class, safe))
  File "C:\Python33\lib\site-packages\peewee.py", line 1223, in create_table_sql
    columns.append(' '.join(foreign_key))
AttributeError: 'map' object has no attribute 'append'


This error affected py3k, which I don't regularly test -- my fault.  I've fixed and re-pushed to PYPI as 2.1.7.
Issue: Missing ChangeLog on PyPi and in release tarballs
Author: pitchforks
State: closed
Created: 2013-12-25 01:00:50

I am using https://pypi.python.org/pypi/peewee to track peewee for new releases. Unfortunately it doesn't include a ChangeLog, so I have to come to https://github.com/coleifer/peewee/releases to read it. The release tarball also doesn't seem to bundle a ChangeLog file.

Would it be possible to have a ChangeLog file in the release tarball, as well as having it listed on https://pypi.python.org/pypi/peewee, for convenience?


That makes sense.  I only recently started tracking changes between versions so I wasn't actually sure anyone even looked at the releases on GH.

Is there a special way to do this, or just make a `CHANGELOG` file in the repo and bundle it with the package?
Issue: Circular references
Author: coleifer
State: closed
Created: 2013-12-23 20:26:12

It would be nice to support circular references between tables:

``` python

class User(Model):
    username = CharField()
    last_post = ForeignKeyField(Post, null=True)  # error

class Post(Model):
    author = ForeignKeyField(User, related_name='posts')
    title = CharField()
```

Self-referential FKs are possible using the magic string `'self'` but this is not possible between tables.


This is partially fixed by d5a5a86a14b7d631ef8b8b3759bfd8345def9b56 which makes it possible to use `Proxy` with `ForeignKeyField`.  Unfortunately, Postgresql will complain about the `create table` queries generated so I'll need to work around this.


http://peewee.readthedocs.org/en/latest/peewee/cookbook.html#circular-foreign-key-dependencies
Issue: Fx places.sqlite not fit for pwiz.py
Author: ricardogithub
State: closed
Created: 2013-12-09 15:04:33

In fact, it is the other way around :-) 

Firefox's **places.sqlite** database (and propably others too) use a slighty modified field model, which is not supported by pwiz.py. 

What I found is that Firefox at least uses non-standard datatypes _long_ and _longvarchar_, and also can use fields _without_ a proper type denotation. 

The _mapping_-array of the SqliteIntrospector of pwiz.py does not support this, and silently emits _UnknownFieldType_ instances in theses cases. 

First. It is easily fixed by adding three lines to the mapping array:
           'long': BigIntegerField, 
           'longvarchar': TextField, 
           '': TextField,

A second thing is: it is usually a good thing not to silently make assumptions, like emitting an UnknownFieldType as mentioned above. The least _pwizz.py_ should do is give a warning.

It would be nice if these things can be added to the script, perhaps only if a special engine flag has been set.


> A second thing is: it is usually a good thing not to silently make assumptions, like emitting an UnknownFieldType as mentioned above.

pwiz emits `UnknownFieldType` because it is unknown...exactly because we don't want to be making assumptions. See: http://www.sqlite.org/datatype3.html .

I don't see any problem with adding `long` -> `BigIntegerField`. I'm not sure about some others.  Maybe it makes sense to put a comment after the `UnknownFieldType` indicating the column type in the table definition.


In my particular case the three additions to the mapping array seem to work, but I have not tested it extensively. 

Good suggestion about the column type hint. 

BTW In general I agree with you about the assumptions, but..._places.sqlite_ happened to be the first database I tried to introspect. I can't believe I was the first one till now to pick places.sqlite :->  

Luckily I stayed on board, but maybe others did not, and moved on to another ORM solution. And that is really too bad, because peewee deserves better than a first crappy experience with pwiz due to some non-standard, but very common database :-) Keep up the good work. 


I'm sorry you were disappointed with the generated code.


I've added some logic to display the raw underlying data type.  With sqlite you can enter in any garbage you want...this is perfectly valid:

``` sql
create table walrus (tusk ivory, whisker blubber);
```

pwiz will now return:

``` python
class Walrus(BaseModel):
    tusk = UnknownFieldType(null=True)  # ivory
    whisker = UnknownFieldType(null=True)  # blubber

    class Meta:
        db_table = 'walrus'
```
Issue: Adds support for nested transactions to postgres via savepoints
Author: jhorman
State: closed
Created: 2013-12-06 18:51:37

Postgres supports nested transactions by way of savepoints.

http://www.postgresql.org/docs/9.3/static/sql-savepoint.html

An example usage is

``` sql
BEGIN;
    INSERT INTO table1 VALUES (1);
    SAVEPOINT my_savepoint;
    INSERT INTO table1 VALUES (2);
    ROLLBACK TO SAVEPOINT my_savepoint;
    INSERT INTO table1 VALUES (3);
COMMIT;
```

This patch adds savepoint support to the peewee PostgresqlDatabase. As is, PostgresqlDatabase doesn't implement begin, or support nested transactions. 

With this change nested transactions will only setup/rollback/release savepoints, the outermost transaction will do the commit/rollback.

begin: now creates a savepoint, and increments the nesting depth.
commit: releases the savepoint, and calls commit if not still nested.
rollback: rolls back to the savepoint, and calls rollback if not still nested.


Interesting PR.  I'm actually looking at refactoring the transaction code to use a stack for #265 , though I did not plan on implementing nested txns using savepoints.

I'm not sure I like overloading `begin` to now mean "create a savepoint".  Same for `rollback`.  These should always mean either begin a transaction or roll it back.


Ok, now I only will create a savepoint in `begin()` for the nested transactions.

In Postgres there is no notion of nested transactions via `BEGIN`. A 2nd call to `BEGIN` will just be ignored. Their recommendation is to create a save point if the intention is to support nesting. That is what the 2nd commit now does. Only nested rollback/commit now release/rollback the savepoints.

This change now supports #265 I believe, on Postgres at least. I think that MySQL would be the same basically, at least with InnoDB.

I like this approach vs having to force the user to think about savepoints, and how nesting is supported across `Database` implementations. This just says that peewee will support nesting, and the details are up to the `Database` implementation.


> I like this approach vs having to force the user to think about savepoints.

That's a valid point.


It looks like MySQL and Sqlite support savepoints as well.  Have you looking into how this PR works with those databases?


Rather than expose savepoints as nested transactions, I've created a new method / context manager.

``` python

with db.savepoint():
    do stuff
```


I like adding explicit savepoint support, but are you saying you don't want to support nested transactions in postgres like this?


Well, I added support for nesting calls to `transaction`, but the transactions are not nested per-se.  I will add support for savepoints as a separate API using an approach similar to the transaction context mgr.


Added savepoint support:

``` python
with db.transaction():
    # etc
    with db.savepoint():
        # etc
        with db.savepoint():
            # etc
```


Cool, I'll check it out.
Issue: It would be nice to support nested transactions within commit_on_success
Author: ricoboni
State: closed
Created: 2013-12-03 12:17:41

Sometimes we have an operation that uses itself the `commit_on_success` decorator but it needs to call another operation using `commit_on_success`. The current version just commits the inner part regardless the erros that may occur in the outer part.

I've solved this problem as bellow, but maybe it worth to have it integrated on peewee. (And maybe better designed)

```
class CustomMySQLDatabase(peewee.MySQLDatabase):
      def commit_on_success(self, func):
          def inner(*args, **kwargs):
              orig = self.get_autocommit()
              if self.autocommit and not orig:
                  # Consideramos aqui que estamos ja dentro de uma transacao
                  return func(*args, **kwargs)
              else:
                  self.set_autocommit(False)
                  self.begin()
                  try:
                      res = func(*args, **kwargs)
                      self.commit()
                  except:
                      self.rollback()
                      raise
                  else:
                      return res
                  finally:
                      self.set_autocommit(orig)
          return inner
```


Great catch, I'm actually surprised this hasn't come up before!


This has been fixed.  See testcase: 54b655c8469fd0f22608ac55f681d99ffab72316


Reopening.


I didn't like my implementation and have re-written it.  New code is in 727299c323f97fb3a5c82c2ca88b1a0adb9ad06f and tests in 318ff7b3ca9689a537999cb923845d6d54014514


Seems very nice. Thank you!
Issue: Type/Range Checking Input?
Author: twodayslate
State: closed
Created: 2013-11-27 13:40:28

Is it possible to do range checking on an insert? For example, when inserting a float it must be between 1.0 and 5.0


Depends on how you're saving that value.  If you're calling `Model.save()` then just override the `save` method and add the check there.  If you're calling `Model.insert()` you may have to implement it as a check constraint in your db.


Does the option argument not do it?


> Does the option argument not do it?

Could you be more specific which option?  Also, how did you plan on inserting these values -- using `Modelsave()` or `Model.insert()`?


I use `Model.create()`

I misspoke. I meant `choices` not `option`. Under the Fields argument list in the API there is a `choices` option. Other libraries I have used have used this as a range feature. `choices=range(1,5)`

I'm in IRC now but I feel like this could help other people as well.


Cool, well since you're using `create()` you can override save:

``` python
class MyModel(Model):
    some_val = FloatField()

    def save(self, *args, **kwargs):
        if self.some_val < 1 or self.some_val > 5:
            raise ValueError('"some_val" must be between 1.0 and 5.0')
        return super(MyModel, self).save(*args, **kwargs)
```

The `choices` argument exists solely as metadata and is not validated (either in python or in the database).  So even if you used an integral range (which itself doesn't exactly mean "any floating point between 1.0 and 5.0") peewee will not complain.
Issue: DDL generation is brittle
Author: coleifer
State: closed
Created: 2013-11-26 22:03:15

Currently the DDL generation code is very brittle.  The upside is that in the simple case it "just works" and isn't much code.

Examples:
- #259 
- #147 

Perhaps the DDL code should be implemented using `QueryCompiler` (or a subclass).


I can finally close this!


What do you mean?


Sorry,  blaming incorrectly. It worked fine.
Issue: strict mode
Author: ricardogithub
State: closed
Created: 2013-11-26 18:10:15

Writing a Model will generate code for the RDBMS of choice, if the create_table() method is called. I noticed that there seems to be no check on the actual response of the RDBMS. See my issue #259, which illustrated that peewee generates a statements for MySQL, but a faulty one, in that particular example. Is there any possiblity to add a sort of 'strictness' switch to peewee? 

This 'strictness' should not only apply to feedback from the RDBMS while generating an index, but also help to avoid misspelled words, e.g. unigue in stead of unique, which was not noticed by me until I discovered that the anticipated index creation was not promoted to the database. 


Won't the database raise an exception in the event you execute a bad query?


I did not mean handling exceptions triggered by the RDBMS, that is working OK, seems. I meant strictness in the parsing and processing of model definitions, e.g. the following attribute definition: 

```
    field = CharField(unigue=True)  # misspelled keyword unique
```

At run-time I did not get a warning or exception from peewee, and peewee will 'just' generate code for the RDBMS. The created table is not as intended, since the unique index is missing. The program seems to work allright, at first glance, but duplicate records may (and in my case will) occur in the table. 

I like Peewee. Great job so far. It may be lightweight, but it should do a proper job :-)

To summarize: more strict parsing of the model is really useful and needed. 


The `kwargs` parameter of a field object's `__init__()` method is being used to populate arbitrary field attributes, so I'm not sure how to add protection against misspellings.


To clarify, since _anything_ could be a valid keyword argument, I'm not sure how to fix this.


A bit challenged by coleifer last remark, I decided to have a look at the peewee code myself. I was struck by the small size of the program, really. It looks good, and as such is a good display of the power of the Python language. 

I don't agree that anything could be a valid keyword. Just put all in a dict, and use that dict to check the passed keyword/value pairs. Just by adding a few lines of code to add to the **init** method of the Field class, just before the invocation of the super-method, enables me to signal my misspellings - at least:

```
    for key in kwargs:
            if key not in ['null', 'index', 'unique' .... 'sequence']:
                    print "Error with passed key: " + key
```

I left out a few keywords here in this pseudo-code fragment. Anyway, it should be referencing a constant list declared somewhere in top of the program. I think there are some more places in the code that could benefit of this little construct.
Issue: unique index on TextField
Author: ricardogithub
State: closed
Created: 2013-11-26 18:01:06

MySQL requires a maximum number of characters that will be relevant for an unique index on longtext fields, whilst the actual field may and will be much larger. Seems this is not possible to define in pw while creating a new table. I am not aware of the same issue in supported rdbms, but is there a possibility to add this now already, or in a future release? 


MySQL is very strange!


I think this should probably be rolled into a larger issue to refactor the DDL generation code.


Closing in favor of 261.