akumoth/peewee-issues huggingface dataset:
Issue: Fix nullable UUID foreign keys
Author: alexlatchford
State: closed
Created: 2014-07-07 06:48:30

Hi guys,

Found a bug in the PostgresExt implementation of UUIDField for foreign keys. Let me know if the patch is satisfactory, it fixes my bug but not sure of the impact, would be great to get this in from my point of view :)

Example:

``` python
class Entity(Model):
    uuid = UUIDField(primary_key=True)
    name = CharField()

class User(Model):
    name = CharField()
    entity = ForeignKeyField(Entity, related_name="users", null=True)


User.create(name="Joe")
user = User.select().where(name="Joe").first()
user.name = "Barry"
user.save() # See error below..
```

Obviously a user **may** belong to a entity. I get this with the current implementation :(

```
Traceback (most recent call last):
 ... <redacted>
  File "/Users/alexlatchford/Code/abbreviate/a8-product/a8_product/../a8_product/database/__init__.py", line 77, in save
    return super(MetaBaseModel, self).save(*args, **kwargs)
  File "/Library/Python/2.7/site-packages/peewee.py", line 3106, in save
    rows = self.update(**field_dict).where(self.pk_expr()).execute()
  File "/Library/Python/2.7/site-packages/peewee.py", line 2140, in execute
    return self.database.rows_affected(self._execute())
  File "/Library/Python/2.7/site-packages/peewee.py", line 1796, in _execute
    return self.database.execute_sql(sql, params, self.require_commit)
  File "/Library/Python/2.7/site-packages/playhouse/postgres_ext.py", line 255, in execute_sql
    self.commit()
  File "/Library/Python/2.7/site-packages/peewee.py", line 2234, in __exit__
    reraise(new_type, new_type(*exc_value.args), traceback)
  File "/Library/Python/2.7/site-packages/playhouse/postgres_ext.py", line 248, in execute_sql
    res = cursor.execute(sql, params or ())
DataError: invalid input syntax for uuid: "None"
LINE 1: ...'c8c4bb1aba234033b57d30b433f03354', "entity_id" = 'None', "i...
```

Many thanks,
Alex

PS. That example should work, my current one is a little bit more complicated than this so I had to write the example on the fly to simplify it down, apologies if it doesn't. You get the gist :)


Looks good, thanks for this!


Awesome cheers Charles. Just me being cheeky but what's your release schedule normally? Is it just when it hits a critical mass or is timed etc., obviously looking to get this into my product officially (I'll use a git dependency for now though :))

Thanks again,
Alex


It's very abstract at the moment, but I can push a new release, no probs.


2.2.5 now available on pypi.


Awesome, thanks! Love this module :smile:
Issue: pylint doesn't see .id members of model instances
Author: pitchforks
State: closed
Created: 2014-07-06 00:16:45

I am not sure this is a peewee issue, so feel free to close it if you think it's invalid (or if you are confident this is a pylint bug)

I am using peewee-2.2.4

Everytime I have a line like myModelInstance.id in the code, pylint complains that "[E1101] Instance of 'MyModel' has no 'id' member".

I don't want to turn this error off globally in the config file of pylint. So, to make pylint shut up, I end with a lot of comments that turn each error off explicitly: "# pylint: disable=E1101" - one such comment at the end of each line where pylint complains. This feels extremely ugly.

Is it possible to make peewee somehow expose this id field, so pylint would stop complaining about it?

edit: typo


I'm not sure what to do about this on the peewee side -- perhaps you could just add the following to your models:

``` python
id = PrimaryKeyField()
```


This works. Thank you.
Issue: How to set threadlocals when using peewee with Flask?
Author: kramer65
State: closed
Created: 2014-07-05 00:01:55

I'm using Peewee with Flask (which is an awesome combination) and because I use threading I want to set `threadlocals=True`. In [the documentation](http://peewee.readthedocs.org/en/latest/peewee/cookbook.html#multi-threaded-applications), it is suggested that I use it like so:

```
concurrent_db = SqliteDatabase('stats.db', threadlocals=True)
```

In my Flask app however, I initialise the DB like this:

```
app = Flask(__name__)
app.config.from_object('config')
db = Database(app)
```

and in the config file I've got among other stuff, this to define the DB:

```
DATABASE = {
    'name': 'mydb',
    'engine': 'playhouse.postgres_ext.PostgresqlExtDatabase',
    'user': 'myuser',
    'password': 'supersecret'
}
```

I tried putting `threadlocals = True` in the config file, which doesn't give any errors, but how do I know whether it really picked it up and is using it? Is there a way to check whether I'm actually using threadlocals?

What is the proper way to set threadlocals to True when using Flask? All tips are welcome!


In your `DATABASE` declaration just put `'threadlocals': True` -- that's all you need to do.
Issue: UUID primary keys
Author: alexlatchford
State: closed
Created: 2014-07-04 03:45:09

I'd like to have `playhouse.postgres_ext.UUIDField` as the primary key for a table.

https://github.com/coleifer/peewee/issues/9 seems to suggest that this should be possible, but seems to have been deprecated since it's first introduction in 2012. Any suggestions?

Cheers,
Alex

PS. Awesome ORM, switched over from SQLAlchemy and loving every minute of it thus far!


Aha, scratch that I found the 1.0 -> 2.0 upgrade notes. My bad, closing...

For anyone finding this:

``` python
class MyModel(Model):
    uuid = UUIDField(primary_key=True)
    ...
```
Issue: Does peewee work with Gevent?
Author: yueyoum
State: closed
Created: 2014-07-02 07:56:38


Yes it does, You need to set Threadlocals to True

```
# app configuration
DATABASE = {
    'name': 'my_db',
    'engine': 'peewee.PostgresqlDatabase',
    'user': 'postgres',
    'threadlocals': True, # <-- this
}
```

and then just monkey patch the thread module.


If you're using it with postgresql you need to monkeypatch `psycopg2`:

``` python
from gevent import monkey

from gevent.socket import wait_read, wait_write
try:
    from psycopg2 import extensions
except ImportError:
    extensions = None


def monkeypatch_psycopg2():
    extensions.set_wait_callback(gevent_wait_callback)

def gevent_wait_callback(conn, timeout=None):
    """A wait callback useful to allow gevent to work with Psycopg."""
    while 1:
        state = conn.poll()
        if state == extensions.POLL_OK:
            break
        elif state == extensions.POLL_READ:
            wait_read(conn.fileno(), timeout=timeout)
        elif state == extensions.POLL_WRITE:
            wait_write(conn.fileno(), timeout=timeout)
        else:
            raise psycopg2.OperationalError(
                "Bad result from poll: %r" % state)
```
Issue: Unable to reach foreign key when doing an 'join on' with table
Author: jonathan-s
State: closed
Created: 2014-06-30 20:44:48

So when I join the ExtensionData and UserOrder together I'm unable to access the foreign key relationship (data_entries) with DailyData. It seems as if it doesn't get the primary key from ExtensionData because of [None] at the end of the SQL query. This does not happen when you the ExtensionData by itself. 

```
output = ExtensionData.select(ExtensionData, UserOrder.avg_user).join(
        UserOrder, on=(ExtensionData.id == UserOrder.link_to_id)
        ).order_by(UserOrder.avg_user.desc())
entry = output[0]
entry.data_entries > 
<class 'models.DailyData'> SELECT t1."id", t1."date_scraped", t1."link_to_id", t1."nr_users", t1."nr_reviews", t1."downloads", t1."rating" FROM "dailydata" AS t1 WHERE (t1."link_to_id" = %s) [None]
entry.id >
<models.UserOrder object at 0x1593490> # id has been replaced by this
```


Try adding `UserOrder.link_to_id` to the select clause:

``` python
(ExtensionData.
 select(
    ExtensionData, UserOrder.avg_user, UserOrder.link_to_id)
 .join(UserOrder, on=(ExtensionData.id == UserOrder.link_to_id))
 .order_by(UserOrder.avg_user.desc()))
```

If that doesn't work, could you share a minimal set of model code so that I can attempt to reproduce the issue?


Nope, your suggested solution did not work. I've shared my model code below. UserOrder is a view. 

```
class BaseModel(Model):
    class Meta:
        database = db

class ExtensionData(BaseModel):
    name = CharField()
    link = CharField(unique=True) 
    hp = CharField(null=True)
    email = CharField(null=True)
    category = CharField(null=True)
    browser = CharField()
    updated = DateField(default=datetime.date.today())

class DailyData(BaseModel):
    date_scraped = DateField(default=datetime.date.today())
    link_to = ForeignKeyField(ExtensionData, related_name='data_entries')
    nr_users = IntegerField()
    nr_reviews = IntegerField()
    downloads = IntegerField(null=True)
    rating = DecimalField()

class UserOrder(BaseModel):
    link_to_id = PrimaryKeyField()
    avg_user = FloatField()

    class Meta:
        db_table = 'user_order'
```


The issue is that the `id` field is getting overwritten by the related `UserOrder`. To fix this:

``` python
(ExtensionData.
 select(
    ExtensionData, UserOrder.avg_user, UserOrder.link_to_id)
 .join(
     UserOrder, 
     on=(ExtensionData.id == UserOrder.link_to_id).alias('user_order'))
 .order_by(UserOrder.avg_user.desc()))
```


Great, thank you. I'm thinking of writing something about this in the documentation, it would be nice to include something about referring views as well as I couldn't find that in the documentation either when I was looking. Perhaps under Peewee cookbook section?


Sounds good, feel free to submit a pull-request.
Issue: MySQL Prefetch with Limit Fix
Author: jpttrssn
State: closed
Created: 2014-06-30 16:11:52

MySQL does not support using LIMIT and IN subquery. To fix you have to run the subquery first and then pass in the result. This will add 1 additional query for every subquery passed to prefetch. This approach causes errors in test_prefetch_non_pk_fk, test_prefetch_subquery and test_prefetch_subquery_same_depth due to there being no database to run the subquery on and fails on test_prefetch_multi_depth and test_prefetch_simple because of the additional queries.

There is a potential work around that would reduce this back to the optimal number of queries by running the outer queries when needed without changing the select, storing the result, and getting the necessary list of foreign keys to use for the inner query result. This would be a decent amount of changes and I defer to you to decide if this is worth it for this specific case and whether this solution is MySQL only. There may be a performance benefit here when you no longer need to run the subquery inside the IN clause, but it could also be worse for larger datasets? Is passing in a list of 1,000 or 10,000 foreign keys to an IN clause problematic? Perhaps there could be a flag that you set on whether you want to use a subquery in the IN clause or a list of results? You would probably know better.


I'm going to pass on merging this changeset. Even though MySQL does not support LIMIT on subqueries, you still could perform the limit prior to calling `prefetch()`, e.g.:

``` python
q1 = User.select()
q2 = Blog.select().limit(10)
prefetch(q1, q2) # does not work on mysql
```

Instead do:

``` python
q1 = User.select()
q2 = Blog.select().limit(10)
q2_fixed = Blog.select().where(Blog.id << [x.id for x in q2])
prefetch(q1, q2_fixed)  # this should work fine.
```


What if I need to limit the User query and not the Blog query?

```
q1 = User.select().limit(10)
q2 = Blog.select()
prefetch(q1, q2) # does not work on mysql
```


Do the same thing...

``` python
q1 = User.select().limit(10)
q1_fixed = User.select().where(User.id << [x.id for x in q1])
prefetch(q1_fixed, Blog.select())
```


I'm trying to avoid running extra queries, but this will do. Thanks, man!
Issue: SQLite migration bug?
Author: coleifer
State: closed
Created: 2014-06-23 13:11:32

``` python
class User(BaseModel):
    id = peewee.CharField(primary_key=True, max_length=20)

class Page(BaseModel):
    id = peewee.PrimaryKeyField()
    name = peewee.TextField(unique=True, null=True)
    user = peewee.ForeignKeyField(User, null=True, related_name='pages')
```

Migration Script:

``` python
from playhouse import migrate

my_db = migrate.SqliteDatabase('database')
migrator = migrate.SqliteMigrator(my_db)

with my_db.transaction():
    migrate.migrate(
        migrator.rename_column('page', 'name', 'title')
    )
```


I can confirm the weird behavior with the testcase above and on my code too. In my case the models are similar: a Feed model and an Entry model, with the latter having a FK to Feed. 

On my machine it gives this error:

> peewee.OperationalError: table page__tmp__ has no column named FOREIGN

I'm using:

```
Underworld:$ python
Python 2.7.6 (v2.7.6:3a1db0d2747e, Nov 10 2013, 00:42:54) 
>>> import sqlite3 as s
>>> s.version
'2.6.0'
>>> s.sqlite_version
'3.7.13'
```


Replicated this in a test-case, working up a fix.
Issue: Is it possible to create a view peewee or join a created query with model?
Author: jonathan-s
State: closed
Created: 2014-06-18 18:18:50

I tried doing this. But got an error. 

```
daily = DailyData.select(DailyData.link_to, fn.avg(DailyData.nr_users).alias('avg_user')).group_by(DailyData.link_to)

output = ExtensionData.select(ExtensionData.name).join(daily).order_by(sql('avg_user DESC')).limit(500)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/scraper/.virtualenvs/superfish/lib/python2.6/site-packages/peewee.py", line 233, in inner
    func(clone, *args, **kwargs)
  File "/home/scraper/.virtualenvs/superfish/lib/python2.6/site-packages/peewee.py", line 1708, in join
    if not self._query_ctx._meta.rel_exists(model_class) and on is None:
  File "/home/scraper/.virtualenvs/superfish/lib/python2.6/site-packages/peewee.py", line 2849, in rel_exists
    return self.rel_for_model(model) or self.reverse_rel_for_model(model)
  File "/home/scraper/.virtualenvs/superfish/lib/python2.6/site-packages/peewee.py", line 2846, in reverse_rel_for_model
    return model._meta.rel_for_model(self.model_class)
AttributeError: 'SelectQuery' object has no attribute '_meta'
```

DailyData.link_to is the foreign key to ExtensionData. Is there another way of doing this?


Ok, I solved it. 

I created a view in my database and then in my models.py I created a model for it. 

```
class UserOrder(BaseModel):
    link_to_id = PrimaryKeyField()
    avg_user = FloatField()

    class Meta:
        db_table = 'user_order'
```

Where db_table is the reference to the actual view and the fields in the view are listed above. Then I could happily join the two together.
Issue: pwiz.py fails for postgres when schemas have cross-dependencies
Author: demongoo
State: closed
Created: 2014-06-18 15:18:27

I have a large PostgreSQL database, with multiple schemas. Some foreign keys involve cross-schema references. Currently `pwiz` just fails with `KeyError` here and there. In my case I've just quick-fixed script to ignore some missing keys and to spit out `UNDEFINED` instead of missing information, but of course it is not a solution.
The problem is that in my case `pwiz` just failed before outputting code, and was useless. Given the size of DB I prefered to have at least some missing Foreign keys, instead of nothing. It may be fixed temporarily by some `--ignore-errors` key, for example.


pwiz certainly has limitations and it's very possible cross-schema foreign keys are one of them. I think I will close this as wontfix but feel free to submit a patch.
Issue: Created signal is not triggered.
Author: soasme
State: closed
Created: 2014-06-17 09:29:37

Signal in playhouse/signal.py:

``` python
def save(self, *args, **kwargs):
        created = not bool(self.get_id())
        pre_save.send(self, created=created)
        super(Model, self).save(*args, **kwargs)
        post_save.send(self, created=created)
```

If I create a model via

``` python
model = Model()
model.id = uuid.uuid4()
model.save(force_insert=True)
```

then created is always `False`, which means that I can never trigger created event.


:+1:
Issue: Row not inserted into table when using CompositeKey
Author: scls19fr
State: closed
Created: 2014-06-14 19:56:06

Hello,

I'm new to Peewee.

I did this code:

```
from peewee import *

db = SqliteDatabase('json_rpc.sqlite')

class RPC(Model):
    terminal_id = CharField()
    request_id = CharField()
    request = CharField()
    response = CharField()

    class Meta:
        database = db
        primary_key = CompositeKey('terminal_id', 'request_id')

# connect to our database
db.connect()

RPC.create_table()

terminal_id = 'test01'
request_id = 2
request = '{"jsonrpc": "2.0", "method": "subtract", "params": [42, 23], "id": '+ str(request_id) + '}'
response = ""
rpc = RPC(
    terminal_id=terminal_id,
    request_id = request_id,
    request=request,
    response=response
)
rpc.save()
```

Unfortunatelly row is not inserted into SQLite database (without any error).

If I comment `primary_key = CompositeKey('terminal_id', 'request_id')`
row is inserted (but there is no security to ensure that there is a unique couple (terminal_id, request_id)).

So I don't know if this is really an issue or a misunderstanding from my side.

Kind regards


Try reading
http://peewee.readthedocs.org/en/latest/peewee/models.html#non-integer-primary-keys
On Jun 14, 2014 2:56 PM, "scls19fr" notifications@github.com wrote:

> Hello,
> 
> I'm new to Peewee.
> 
> I did this code:
> 
> from peewee import *
> 
> db = SqliteDatabase('json_rpc.sqlite')
> 
> class RPC(Model):
>     terminal_id = CharField()
>     request_id = CharField()
>     request = CharField()
>     response = CharField()
> 
> ```
> class Meta:
>     database = db
>     primary_key = CompositeKey('terminal_id', 'request_id')
> ```
> 
> # connect to our database
> 
> db.connect()
> 
> RPC.create_table()
> 
> terminal_id = 'test01'
> request_id = 2
> request = '{"jsonrpc": "2.0", "method": "subtract", "params": [42, 23], "id": '+ str(request_id) + '}'
> response = ""
> rpc = RPC(
>     terminal_id=terminal_id,
>     request_id = request_id,
>     request=request,
>     response=response
> )
> rpc.save()
> 
> Unfortunatelly row is not inserted into SQLite database (without any
> error).
> 
> If I comment primary_key = CompositeKey('terminal_id', 'request_id')
> row is inserted (but there is no security to ensure that there is a unique
> couple (terminal_id, request_id)).
> 
> So I don't know if this is really an issue or a misunderstanding from my
> side.
> 
> Kind regards
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/coleifer/peewee/issues/374.


Thanks for the tip
`rpc.save(force_insert=True)` fixes my problem
Issue: Row not inserted into table when using CompositeKey in Meta Model
Author: scls19fr
State: closed
Created: 2014-06-14 19:32:22

Hello,

I'm new to Peewee.

I did this code:

```
from peewee import *

db = SqliteDatabase('json_rpc.sqlite')

class RPC(Model):
    terminal_id = CharField()
    request_id = CharField()
    request = CharField()
    response = CharField()

    class Meta:
        database = db # this model uses the people database
        primary_key = CompositeKey('terminal_id', 'request_id')

# connect to our database
db.connect()

RPC.create_table()

terminal_id = 'test01'
request_id = 2
request = '{"jsonrpc": "2.0", "method": "subtract", "params": [42, 23], "id": '+ str(request_id) + '}'
response = ""
rpc = RPC(
    terminal_id=terminal_id,
    request_id = request_id,
    request=request,
    response=response
)
rpc.save()
```

Unfortunatelly row is not inserted into SQLite database (without any error).

If I comment `primary_key = CompositeKey('terminal_id', 'request_id')`
row is inserted (but there is no security to ensure that there is a unique couple (terminal_id, request_id)).

So I don't know if this is really an issue or a misunderstanding from my side.

Kind regards
Issue: playhouse.migrate should use Database.compiler_class when possible
Author: stas
State: closed
Created: 2014-06-14 18:02:00

There're some inconsistencies between database compiler and migrator methods.
A good example is the `add_index` method where when using Postgres, array and hstore fields won't get gist index types.

Delegating these methods to the compiler class fixes the issue.
This will also reduce the duplication.


Good catch, thanks I will look into fixing this.


The thing is that the migrator methods for adding indexes are not aware of the column types, so this doesn't really make sense. Maybe it would be better to subclass `Migrator` and allow the `add_index` operation to accept some additional arguments?

For now I am closing as wontfix.


You're right, the problem is that migrator has no idea about the column type. As a solution right now, I'm calling the `Model._meta.database.add_index` instead. Tbh, I would just make the migrator method do the same: pass the call to the existent database instance method.
Issue: Resolved issues with using peewee on app engine.
Author: jpttrssn
State: closed
Created: 2014-06-07 21:37:06

App Engine does not support sqlite3 so in order to get peewee to work I had to add in checks for it. This was my quick and dirty solution. There's probably a more elegant way to handle this.


I think I might take a crack at this myself. I'm surprised that there's no sqlite on GAE!


Fixed in 6838d8ec54093ebefca4b436abff601e14f94795 -- thanks for the diff and bringing this to my attention! Look forward to more PRs in the future!
Issue: Model.__getattribute__ 
Author: jonathan-s
State: closed
Created: 2014-06-03 12:10:03

The following works as expected:

```
prop = 'nr_users'
datapoints = entry.data_entries.order_by(DailyData.date_scraped.desc()).limit(10)
data = data + [str(entry.__getattribute__(prop)) for entry in datapoints]
```

However the following does not work. I'm not sure if that is expected or not. But I expected it to work. Instead this returns `TypeError: expected 1 arguments, got 0`:

```
output = ExtensionData.select().join(DailyData).order_by(
        DailyData.__getattribute__(prop).desc()).limit(500).iterator()
```


Where is the `TypeError` originating from? Can you include the full traceback?


Actually I think I know what might be happening.

Try changing your `__getattribute__()` to `getattr`, e.g.

``` python
ExtensionData.select().order_by(getattr(DailyData, prop).desc()).limit(500).iterator()
```


Thanks!
Issue: How to get foreign key column itself?
Author: ayang
State: closed
Created: 2014-06-01 12:35:32

Example for this:
class Person(Model):
    name = CharField()
    birthday = DateField()

class Pet(Model):
    owner = ForeignKeyField(Person, related_name='pets')
    name = CharField()

I want to get pet.owner_id, but I can only use pet.owner.id, which will execute a sql query extraly.


You can use `Pet._data['owner']`.


Thank you