akumoth/peewee-issues huggingface dataset:
Issue: fix when not using named cursor
Author: eseom
State: closed
Created: 2015-05-04 17:53:40

I'm using peewee on my projects. Thanks. :)

I wrote a simple connection to the postgresql server without named cursor.

```
from peewee import *  # NOQA
from playhouse.postgres_ext import *  # NOQA

database = PostgresqlExtDatabase('test', user='test')


class MyModel(Model):
    name = CharField()
    value = CharField()

    class Meta:
        database = database


MyModel.create_table()
```

An error occurred.

```
Traceback (most recent call last):
  File "test.py", line 15, in <module>
    MyModel.create_table()
  File "/home/vagrant/src/peewee/peewee.py", line 4010, in create_table
    db.create_table(cls)
  File "/home/vagrant/src/peewee/peewee.py", line 3115, in create_table
    return self.execute_sql(*qc.create_table(model_class, safe))
  File "/home/vagrant/src/peewee/playhouse/postgres_ext.py", line 356, in execute_sql
    cursor = self.get_cursor()
  File "/home/vagrant/src/peewee/playhouse/postgres_ext.py", line 343, in get_cursor
    return self.get_conn().cursor(name=name)
TypeError: argument 1 must be string, not None
```


Interesting, what version of psycopg2 are you using?


missed my environment
ubuntu 14.04
python 2.7.6
psycopg2 is 2.4.5 from python-psycopg2 package

It could be this situation.

```
import psycopg2

name = None
connection = psycopg2.connect('dbname=test user=test')
cursor = connection.cursor(name=name)
```

```
Traceback (most recent call last):
  File "test.py", line 5, in <module>
    cursor = connection.cursor(name=name)
TypeError: argument 1 must be string, not None
```


Ew, just a tip but you should definitely be using pip and not the Ubuntu psycpog2 package.

That said, seems like a fine PR so I'll go ahead and merge.
Issue: Default for register_hstore in postgresql extensions should be False
Author: elgow
State: closed
Created: 2015-04-30 23:57:09

Default usage of the pewee playhouse postgresql extension fails to work with an out-of-the-box postgresql database. The postgresql database ships with the hstore extension not installed, but the playhouse postgresql extension has this default setting.  
    kwargs.pop('register_hstore', True)

This causes the DB to fail to open if the user has not explicitly installed the hstore extension in their database.  This is confusing if the user does not intend to use hstore. 

At very least, this deserves a warning box in the documents to alert the user to open the PostgresqlExtDatabase with register_hstore=False unless the hstore extension is already installed in the database that they intend to use.
Issue: Closing MySQL Database and / or Model creation decorator
Author: conqp
State: closed
Created: 2015-04-30 09:10:55

Hi coleifer,

I am working with peewee now for some time - mostly with MySQL databases - and have encountered several problems regarding the lack of automatic closing of database connections, particularly Error 2006 as described here: http://peewee.readthedocs.org/en/latest/peewee/database.html#using-mysql
I have modified the `peewee.MySQLDatabase` accordingly to always use execution contexts when handling queries.
This resolved the aforementioned issue for me and I do not need to worry about connection management (which is unlikely what you want to do when using an ORM).
Additionally I added a module-level function to be used as a decorator to create modules automatically on loading their respective module.
Below you'll find the patch for commit 911833b1b08a44041cf99cdc76117adec1a6832b

```
29a30
> from contextlib import suppress
3473a3475,3480
>     def execute_sql(self, sql, params=None, require_commit=True):
>         """Executes an SQL query with an explicit connection"""
>         with self.execution_context():
>             return super(MySQLDatabase, self).execute_sql(
>                 sql, params=params, require_commit=require_commit)
> 
4159a4167,4181
> 
> 
> def create(model):
>     """Decorator for peewee.Model definitions that
>     actually should be created on load.
> 
>     Usage:
>         @create
>         class MyModel(peewee.Model):
>             pass
>     """
>     with suppress(OperationalError):
>         with model._meta.database.execution_context():
>             model.create_table(fail_silently=True)
>     return model
```

I'd be glad if you consider my patch for upstream or give me some feedback why you wouldn't.

Cheers

Richard alias coNQP


I prefer to keep model creation APIs as-is. Of course if you like using this function to create tables at module-load time then you could release it as a peewee extension. I prefer to avoid any import-time side-effects and APIs that promote them.


Thanks for the feedback. But what about the first part of the patch regarding auto-closing MySQL connections?
Patch for 911833b1b08a44041cf99cdc76117adec1a6832b

```
3473a3474,3479
>     def execute_sql(self, sql, params=None, require_commit=True):
>         """Executes an SQL query with an explicit connection"""
>         with self.execution_context():
>             return super(MySQLDatabase, self).execute_sql(
>                 sql, params=params, require_commit=require_commit)
> 
```
Issue: Class methods on model instances (why?)
Author: foxx
State: closed
Created: 2015-04-24 15:07:50

As per [documentation](1), models have class methods such as `get_or_create`, which can be used to create a new model instance. Merging together object and objectset methods doesn't feel like a clean abstraction (imho), e.g. the ability to `get` another object which is unrelated to the object we're calling `get` from. For example;

```
u = User.get_or_create(name='amber')
u.get_or_create(name='jessica')
```

Previously I've always followed the design approach of separating object and objectset, similar to the approach that [Django](2) uses. For example;

```
u = User.objects.get_or_create(name='amber')
```

Could you give a brief explanation behind your reasoning for having methods such as `get_or_create` as class methods on the object, rather than a method on a manager class (such as `User.objects`)? Is this a legacy design thing that won't change due to backwards compatibility, or is there some technical merit behind this approach? 

This is more for my own knowledge/learning, as it's left me wondering if my design principles are flawed or if I've overlooked some detail, so your insight would be appreciated.

Many thanks


Classmethods operate on the table. Instance methods operate on the rows. Class represents table, instance represents row. It is just an inconvenient fact of python's `classmethod` decorator that you can call classmethods from an instance. I could write a special decorator that prevented this, e.g. `classonlymethod` but I don't think it's worth the time and more Pythonic anyways.


That makes sense, I had considered using a special decorator myself in the past but it always felt like a bit too much magic, and made it confusing for anyone new to the code. This has given me some food for thought anyway, thanks again for the quick reply.
Issue: connect() doesn't allow sqlite://:memory:
Author: foxx
State: closed
Created: 2015-04-24 12:56:43

The following code does not work;

``` py
db = connect('sqlite://:memory:')

  File "/usr/lib/python2.7/urlparse.py", line 113, in port
    port = int(port, 10)
ValueError: invalid literal for int() with base 10: 'memory'
```

This is because `urlparse` doesn't see the above as a valid connection string, and rightly so. One option would be to check for this specific use case and handle accordingly [here](1), however this doesn't feel very clean and would force people to build invalid URIs.

```
def connect(url):
    driver, target = url.split(":", 1)
    if driver in ('sqlite', 'sqliteext'):
        if target == ':memory:':
            return schemes[driver](':memory')
```

It would seem `SqliteDatabase` already uses `:memory:` if no connection string is provided, and as such the following will work fine;

``` py
db = connect('sqlite://')
```

Although this feels like strange behaviour in someways, it's actually correct because `sqlite://:memory:` is not a valid connection strings, as mentioned earlier. Perhaps a docs update for `connect()` would be most appropriate here?


The correct way is to write:

``` python
connect('sqlite:///:memory:')
```


Doh, you are correct. Confirmed working, my apologies.
Issue: Lastly... thank you
Author: foxx
State: closed
Created: 2015-04-24 11:40:50

I've raised quite a few issues today, but I'd just like to take a moment to say a huge thank you to everyone involved in getting peewee where it is today. The code appears to be well written, I was able to easily follow the source code and understand what was happening, and overall is just a pleasure to use. 


Glad you're enjoying using peewee!
Issue: Database router and connection manager support
Author: foxx
State: closed
Created: 2015-04-24 11:37:53

As an extension from #582, although Playhouse has an existing class for [ReadSlave](3), it doesn't appear to be a suitable replacement for database routers, as seen in [django](4). 

Admittedly adding support for database routers would most likely require some sort of connections manager, and would need some careful thought on design decision. For example, being able to automatically create/teardown test databases based on a database settings dictionary, similar to the [test db feature](5) in Django.

If such a feature would ever be considered, I will happily spend some time putting together a PR proposal, and would welcome comments/ideas before hand.

Thoughts?


Extending the discussion from #582, I was unable to find a clean way to introduce module inspection into Peewee without also bringing in the concept of 'apps', which is way outside of its scope as an ORM.

The cleanest way I've found so far is to use a connection manager by wrapping each connection with a proxy and referencing it later on. The test case can then tell the database proxy to switch out the current connection for the test database connection.

However this breaks because `create_model_tables()` also requires models to be defined explicitly, and due to module inspection not being a clean way forward, the next alternative would be to maintain some sort of global registry of models. This could perhaps be achieved by using the `BaseClass` metaclass to place itself into a list on the peewee module.

Further more, `create_model_tables()` is unable to distinguish which classes are "base inheritance" classes, e.g. used by other models but not actually a model themselves. The best option I can think of would be to introduce a new meta option called `abstract`, similar to the one in [Django](1), which would prevent this problem from happening.

Naturally this would be quite a big change but none of these would be backwards incompatible. The proposal so far would be for;
- Database router
- Connection manager
- Models registry
- New meta option `abstract`


In order to keep peewee simple I have chosen not to implement this type of functionality. The database is simply an attribute on the `Model._meta` object, so tooling could be built to change this at run-time, but at no point do I think peewee will include routers/connection managers like Django for example.


Thanks for the quick reply, appreciated.
Issue: test_database requires explicit list of models
Author: foxx
State: closed
Created: 2015-04-24 11:35:52

If you wish to use `test_database`, you have to give it an explicit list of models, as seen [here](1). This can lead to some surprising behaviour if you create a model class and forget to add it to the list.

One way around this, as seen [here](2), is to use module inspection to auto detect classes without having to explicitly define them. Now this may not be a suitable option for everyone, for example if your models are split across multiple databases.

My original proposal was to add module inspection into `test_database()`, however further testing of #583 has shown this is not feasible. Realistically there isn't much that can be done to improve this unless #583 is accepted, so I'm going to mark as closed.


As a temporary workaround, I was able to do the following;

``` py
import unittest
import inspect
import models
import peewee
from app import testdb
from playhouse.test_utils import test_database

class PeeweeTestDatabaseMixin(object):
    def run(self, *args, **kwargs):
        classes = [ v for k,v in inspect.getmembers(models, inspect.isclass) 
            if issubclass(v, peewee.Model) ]
        test_database(testdb, classes)
        return super(PeeweeTestDatabaseMixin, self).run(*args, **kwargs)
```


I prefer keeping things explicit as it is easier to understand and the implementation is simpler.
Issue: db_url.connect does not support pooling
Author: foxx
State: closed
Created: 2015-04-24 10:44:45

As seen [here](1), there is no way to make `connect()` use a pooled connection. I've looked over the code and there doesn't appear to be any way to convert an existing DB object into a pooled one, at least short of using some horrible metaclass hacks or monkey patching.

Could someone clarify if this feature is on the roadmap, or whether a PR would be accepted if not?


In the mean time, for anyone who comes across this problem in the future, I was able to get a temporary monkeypatch working;

``` py
from playhouse import db_url
from playhouse.pool import PooledMySQLDatabase
db_url.schemes['mysql'] = PooledMySQLDatabase
```

The above will replace the MySQL backend with a pooled backend, whenever `connect()` is used. Replace classes accordingly for other databases, see [here](1).


You can use `db_url.parse(url)` to parse the data into a dictionary, then pass the dictionary values into a pooled database implementation. Because pooled databases require additional arguments (pool size, etc), they do not seem to me a good candidate for inclusion in the db_url module.


That works beautifully, thank you.
Issue: JSON support with MariaDB
Author: foxx
State: closed
Created: 2015-04-24 09:39:47

It would seem that Playhouse has [support](1) for Postgres JSON, where as it doesn't have the same for MySQL. MariaDB, which is backwards compatible, has [supported](1) JSON since 10.0.1 and it would be fantastic to see Playhouse support for this.

Are there any plans to put this on the roadmap?


I have no plans myself for adding support for any MySQL-specific features since between Postgres and SQLite I never find myself using MySQL. That said, I would certainly be open to a well-tested pull-request (with documentation).


Thanks for the quick reply, if I end up using MariaDB JSON heavily then I'll put together a PR.
Issue: let the Meta class can extends from another class
Author: anjianshi
State: closed
Created: 2015-04-22 01:19:05

Example:

``` python
db = PostgresqlDatabase("mydb")

class BaseMeta:
    database = db

class BaseModel(Model):
    class Meta(BaseMeta):
        pass

class ModelA(Model): 
    pass

class ModelB(Model):
    class Meta(BaseMeta):
        db_table = "model_b"
```


I can see the reasoning behind this change, but I'm loth to introduce any changes that might break existing applications. I'm not sure off-hand, but given the way peewee munges `Meta` objects at present, there may be some edge-cases or situations where this breaks.

What do you think?


I see, the `dir()` function may be unstable.
Can we use `inspect.getmembers()` instead? 
http://stackoverflow.com/a/8529470/2815178
I think it's more safety.

``` python
import inspect
for k, v in inspect.getmembers(meta):
    if not k.startswith('_'):
        meta_options[k] = v
```


Or I can just use a decorator:

``` python
def with_database(cls):
    cls.database = db
    return cls


class BaseModel(Model):
    @with_database
    class Meta:
        pass

class ModelA(Model): 
    pass

class ModelB(Model):
    @with_database
    class Meta:
        db_table = "model_b"
```

Then, nothing need changes in Peewee.
It's just not so nature.


You do know you can write this, right?

``` python
db = PostgresqlDatabase("mydb")

class BaseModel(Model):
    class Meta:
        database = db

class ModelA(BaseModel): 
    # no need to define Meta / database = db, it is inherited.
    pass

class ModelB(BaseModel):
    class Meta(BaseMeta):
        # database = db --> this is inherited via BaseModel
        db_table = "model_b"
```


Oh, I don't know this before. Thank you!
Perhaps this pull request is not needed any more?
Issue: peewee.DataError: (1406, "Data too long for column ...
Author: martinburch
State: closed
Created: 2015-04-21 16:48:21

I'm trying to create a custom field type

```
class JSONField(Field):

    def db_value(self, value):
        return json.dumps(value)

    def python_value(self, value):
        return json.load(value)
```

And then use this to write some data. But I'm being told its too long for the column. How do I explain to peewee what size of data the column can accept?

Edit: I have also tried changing to `class JSONField(TextField):` instead of just `Field` but no change in the error.


Sorry, I think this was a database issue, peewee was just passing back the error.
Issue: peewee successfully updates model with existed unique fields
Author: semolex
State: closed
Created: 2015-04-16 14:57:29

I have such code:

``` python
class User(peewee.Model):
    username = peewee.CharField(unique=True, max_length=50)
    password = peewee.CharField()
    email = peewee.CharField(unique=True, max_length=80)
    status = peewee.IntegerField(choices=[(1, 'active'), (2, 'inactive'), (3, 'blocked')], default=1)
    class Meta:
        database = database
```

way of creating table:

``` python
User.create_table()
```

I have already created record in my DB with username 'Alex'.
And after that, when I'm trying to update some model with username 'Alex', it is updates successfully for the first time! It fails only when I try to do this twice!
Maybe it is issue with something else?


I'm not sure I understand what is happening. Can you share code to replicate the issue?


I feel sorry for disturbing you. It is not issue of peewee itself, it was at DB level...
Issue: Support encoding in Postgres
Author: klen
State: closed
Created: 2015-04-16 10:22:45

Hello,

In my case I work with a lot of unicode data. All my databases have a UTF-8 encoding, but I've got errors like this: `UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-5: ordinal not in range(128)`.

I knew about the trick:

``` python
database = PosgresqlDatabase(database, **connection_params)
conn = database.get_conn()
conn.set_client_encoding('UTF8')
```

But if you have multiple workers (or pool) it doesn't help and the error still appears. 

So I provide the pull request.

```
database = PostgresqlDatabase(database, encoding='UTF8', **connection_params)
```

Which solve the problem completely.


Thanks!


Thank you! Can I hope on patch-release?


I'll probably wait a bit as I just released a new one not long ago. Maybe a couple weeks?


Up to you. But for now I have to use manual installation from github, it's not compatible.


I just released 2.6.0.


Great! Thank you!
Issue: postgres_ext do not support LTreeField?
Author: dllhlx
State: closed
Created: 2015-04-16 09:24:13

from the docs <http://docs.peewee-orm.com/en/1.0.0/peewee/playhouse.html#ltree > we can know peewee ext can support postgres ltree but in the source code in github we can not find the define of LTreeField. why?


i find define of LTreeFiled in peewee old version - 0.9.9 , why delete this logic in lastest version ?


> from the docs we can know peewee ext can support postgres ltree

Where in the docs does it say this?


I have no intentions of adding LTree back, but would accept pull-requests.
Issue: PostgerSQL:how to add explicit type casts?
Author: khahux
State: closed
Created: 2015-04-15 20:58:42

```
@classmethod
def get_blog_by_date(cls, year, month):
    format = '%%%s-%s%%' % (year, month)
    return cls.select(cls.created_at).where(cls.created_at ** format)
```

------ error ------
    ProgrammingError: operator does not exist: timestamp without time zone ~~\* unknown
    LINE 1: ...ed_at" FROM "blog" AS t1 WHERE ("t1"."created_at" ILIKE '%20...
                                                                 ^
    HINT:  No operator matches the given name and argument type(s). You might need to add explicit type casts.

I find this   select \* from events where CAST(timestamp as TEXT) like '2010-01-26 10:%'; http://www.question-defense.com/2010/02/08/no-operator-matches-the-given-name-and-argument-types-you-might-need-to-add-explicit-type-casts
but not solve this problem by searching.


Hey, if you are using the `PostgresqlExtDatabase` you can use `cls.created_at.cast('text') ** format`.

Honestly why on earth are you doing your date comparison like that, though?! You can use the `between()` function, `fn.date_trunc()`, so many alternatives.

For example, I believe this should work:

``` python
@classmethod
def get_blog_by_date(cls, year, month):
    return cls.select(cls.created_at).where(
        fn.date_trunc('month', cls.created_at) == datetime.date(year, month, 1))
```
Issue: A more powerful get_or_create method
Author: lsc20051426
State: closed
Created: 2015-04-15 07:17:42

Currently the get_or_create is pretty weak.
Here is the example from django:

```
try:
    obj = Person.objects.get(first_name='John', last_name='Lennon')
except Person.DoesNotExist:
    obj = Person(first_name='John', last_name='Lennon', birthday=date(1940, 10, 9))
    obj.save()
```

This pattern gets quite unwieldy as the number of fields in a model goes up. The above example can be rewritten using get_or_create() like so:

```
obj, created = Person.objects.get_or_create(first_name='John', last_name='Lennon',
                  defaults={'birthday': date(1940, 10, 9)})
```

Thanks~


I think I will pass on making changes to the implementation. The method has been deprecated for some time and there is documentation on implementing it yourself:

http://docs.peewee-orm.com/en/latest/peewee/querying.html#get-or-create


These changes are now included in the new version 2.6.0. The function behaves just like it's Django equivalent.


Cool!
Issue: joining is not allowed on UpdateQuery instances
Author: arski
State: closed
Created: 2015-04-14 18:28:18

this is an error message from peewee. May I ask why this is though? Let's assume I want to run an update on Books where the Publisher of said book is based in the US. That's imo a very common update query. Or is there just a different way of doing this during an update?


``` python
us_pub = Publisher.select().where(Publisher.country == 'US')
Book.update(foo='bar').where(Book.publisher << us_pub)
```

IIRC only MySQL supports joins on update queries, while SQLite and Postgres do not. Subqueries are the way to go.


aha! thanks for the quick reply.


actually, what if i wanted to populatw my books with some value
from the related publisher.. No way to do that i guess?


If you're using postgres you can theoretically use `update({Book.foo: Publisher.bar}).from_(Publisher)`

@GothAck is correct, and this is supported by peewee. Sqlite will be adding support for this in the next release as well, which is exciting.

Example peewee usage:

https://github.com/coleifer/peewee/blob/611a08987e8f08af1056ec679ea6cfdccb87c65b/tests/model_sql.py#L689-L743
Issue: Expose parsed db url
Author: stt
State: closed
Created: 2015-04-02 18:38:59

It'd be useful to get the parsed dict from db_url so it could be used with pooled connections as well.
E.g.:

```
db = PooledMySQLDatabase(
    max_connections=32,
    stale_timeout=300,  # 5 minutes.
    **db_url.parse(app.config['DATABASE_URL'])
    )
```


Nice! Do you mind adding a test-case for the new function?


Sure, :+1: ?


Thanks!