akumoth/peewee-issues huggingface dataset:
Issue: http://www.peewee-orm.com/ gives "Not found."
Author: kramer65
State: closed
Created: 2016-02-22 12:41:27

Although I know the docs can be found at http://docs.peewee-orm.com/ , it looks pretty crappy if the root of a website gives an Error.

Maybe it would be good to redirect http://www.peewee-orm.com/ to http://docs.peewee-orm.com/

Cheers!


Actually it used to do that so I was surprised by your note. I went in and looked at the nginx config and realized I hadn't updated it for when I removed Varnish from my site. The redirects should be working now. Thank you!
Issue: Documentation, code example for webapp2
Author: onel
State: closed
Created: 2016-02-20 17:11:14

I know webapp2 is mainly used with app engine, but I started using it ouside of GAE and with peewee.
You can add the documentation for the db opening, closing on each request

``` python
class BaseHandler(webapp2.RequestHandler):

    def dispatch(self):

        db.connect()

        try:
            # Dispatch the request.
            webapp2.RequestHandler.dispatch(self)
        finally:
            if not db.is_closed():
                db.close()
```


Going to pass as this is a pretty obscure framework.


We use this framework (webapp2) with peewee.  We never call connect() or close(). Is that bad?

@savraj -- Probably... I don't know webapp2, but my guess is that you aren't getting the concurrency you think you might be if you're running in certain types of environments.

If you're using a webapp, my opinion is that you should always open a conn upon receiving a connection and close it afterwards. You *can* use a connection pool if you get a lot of traffic, but that's how I would do things.
Issue: RHEL python 3.4 Unable to create SQLITE db
Author: sripathivenky
State: closed
Created: 2016-02-20 03:57:59

Keep running into 

> Traceback (most recent call last):
>   File "db_test.py", line 2, in <module>
>     from playhouse.sqlite_ext import SqliteExtDatabase
>   File "/home/srujan/pyats/pyats/lib/python3.4/site-packages/playhouse/sqlite_ext.py", line 74, in <module>
>     FTS_VER = sqlite3.sqlite_version_info[:3] >= (3, 7, 4) and 'FTS4' or 'FTS3'
> AttributeError: 'NoneType' object has no attribute 'sqlite_version_info'

Upgraded the sqlite3 version to : 3.11.0 2016-02-15 17:29:24 3d862f207e3adc00f78066799ac5a8c282430a5f 

from peewee import *
from playhouse.sqlite_ext import SqliteExtDatabase
import datetime

db = SqliteExtDatabase('my_database.db')

class BaseModel(Model):
    class Meta:
        database = db

class User(BaseModel):
    username = CharField(unique=True)

class Tweet(BaseModel):
    user = ForeignKeyField(User, related_name='tweets')
    message = TextField()
    created_date = DateTimeField(default=datetime.datetime.now)
    is_published = BooleanField(default=True)


No idea, but somehow your install isn't finding a python sqlite driver.
Issue: Can't create index on TextField. Database is MySQL
Author: tgrant59
State: closed
Created: 2016-02-19 20:46:57

When I run create_table() on a model with a TextField(index=True) column, it fails with this error:

```
peewee.OperationalError: (1170, "BLOB/TEXT column 'keywords' used in key specification without a key length")
```

The table is created successfully, but the index is not.
Here's a simple example which reproduces the error using my setup:

``` python
import peewee
from playhouse.pool import PooledMySQLDatabase

class TestModel(peewee.Model):
    keywords = p.TextField(index=True)

    class Meta:
        database = PooledMySQLDatabase("testdb", user="user", passwd="password")

TestModel.create_table()
```

```
peewee==2.7.4
MySQL-python==1.2.5
mysql  Ver 14.14 Distrib 5.7.9, for osx10.10 (x86_64) using  EditLine wrapper
```


Yeah, that's a limitation of MySQL -- you need to specify a length to index when you have a `TextField`.
Issue: 2.8.0 breaks custom Field subclasses that don't check for None
Author: josefdlange
State: closed
Created: 2016-02-18 20:00:26

This is a lot of conjecture (and I'm moving at a pace too quick to really reproduce and test this out), but I'm pretty sure 2.8.0 breaks a Field subclass defined like this:

```
class MyField(Field):
    def python_value(value):
        return some_thing_that_cant_take_none(value)
```

I had to change a `Field` subclass of mine to work like so:

```
class MyField(Field):
    def python_value(value):
        if value is None:
            return None
        return some_thing_that_cant_take_none(value)
```

Not a big deal to me in the long run, but if it's a field that is fundamental to a data model it's a rather obscure issue.


My best guess is that a DB value of `None` would, in prior versions, never get passed to the `Field` subclass's `python_value` method but instead short-circuit to just returning `None`.

This is kind of a breaking change that isn't noted in the `CHANGELOG` if it proves to be what I'm experiencing. Granted I should have `None`-checked anyway but  Figured this is a good word of warning to anyone in the future.


Might be related to #791 actually; perhaps my `python_value` method was bad all along and just never being called because of this bug! These fields are PKs of tables....


I'm pretty sure it's been that way for a while. Not an issue, IMO.


Fair enough. Moving from 2.7.4 --> 2.8.0 broke this for me, but so long as the solution is to fix my own f*cking code I can live with that :)
Issue: Using << with an empty list creates an invalid SQL query
Author: Prillan
State: closed
Created: 2016-02-18 09:49:52

```
psycopg2.ProgrammingError: syntax error at or near ")"
```

I think either of the following solutions are acceptable:
- Throw an error in peewee indicating that the list (iterator) is empty.
- Transform `column << []` into `False` before compiling to SQL.

Not handling this before sending it to the database causes all queries after it to fail (at least on our flask server) with error:

```
peewee.InternalError: current transaction is aborted, commands ignored until end of transaction block
```


Rather than modify the SQL behind-the-scenes, you should check your list at run-time.


I get that, I'm fine with checking if the list is empty. I'm just against creating an invalid query since it results in a syntax error. The error should be thrown before it gets to the database and not in it. To me this is like having a compiler generate invalid code, something that shouldn't be figured out when you try to run the code, but rather at compile time.
Issue: Get extra connection args from query string in db_url
Author: RealSalmon
State: closed
Created: 2016-02-18 02:18:22

### Changes
- As discussed in #858, this adds support for supplying extra connection args in the query string when using playhouse.db_url


Looks good, just one thing to change and I'll be happy to merge.


Actually that dictionary comprehension breaks in Python 2.6 (forgot about that). I didn't notice it before because travis didn't pick up the PR initially.


A few fixes were required for Python 2.6.
Issue: Add support for additional connect args as query string in db_url
Author: RealSalmon
State: closed
Created: 2016-02-17 21:05:04

I'm happy to take a stab at this, but I thought it best to first propose it prior to spending any time . . . 

When using playhouse.db_url for connections, it is not currently possible to include additional connect args directly in the connection string. These must instead be supplied as separate keyword args.

``` python
connect('mysql+pool://user:passwd@ip:port/my_db', max_connections=20, stale_timeout=300)
```

It would be nice to get those extra connection arguments directly from the connection string itself

``` python
# DATABASE_URL has mysql+pool://user:passwd@ip:port/my_db?max_connections=20&stale_timeout=300
connect(os.environ['DATABASE_URL'])
```

Thoughts?


Yes!


Great, I'll start messing around with it this evening (real life permitting . . . ). 


That's cool but how could we set False (eg. register_hstore) via url? Should parse the query string with `keep_blank_values=True`, right?
Issue: deferred pool
Author: coleifer
State: closed
Created: 2016-02-17 20:22:00

I have a question regarding run-time database configuration as described here: http://docs.peewee-orm.com/en/latest/peewee/database.html#run-time-database-configuration

Is this technique expected to work with the connection pooling extensions in playhouse.pool? e.g. . . . 

   db = PooledMySQLDatabase(None)

I end up getting a TypeError exception (TypeError: **init**() got an unexpected keyword argument 'stale_timeout') when I call init() because of the pooling-specific arguments (max_connection, stale_timeout).


dc258ae727620c01d300c67ab58a73c9d1a15b7e
Issue: Possibility to see what changes with a save?
Author: kramer65
State: closed
Created: 2016-02-15 20:25:36

Let's say I've got a `User` model from which I start a new record as follows:

```
u = User()
u.username = 'bob'
u.location = 'home'
u.save()
```

I now use the same user instance and write two values of which one is the same:

```
u.username = 'bob'  # stays the same
u.location = 'work'  # changes
u.save()
```

When saving the updated values, is there a way with Peewee to know which of the values changed (i.e.: `location`), without actually comparing them with the existing values?

Ive got a model with about 50 fields to which I simply save the values of a json object. It would be so much easier if I could get the differences, instead of me needing to compare all those 50 values manually.


Well, peewee knows that you set attributes so those are stored in the `dirty_fields` attribute. But there's no checking to see if it differed from the previous value. Any additional logic will have to live in your app itself.


You can do `model.save(only=model.dirty_fields)`, if you just want to save those that were "set", but it doesn't track actual changes to the underyling value.


Alright thanks. In that case I will need to build something myself.

Thanks!


Why wouldn't peewee check whether values have actually changed? Speed?

And would this be a good way to implement it?

```python
class BaseModel(Model):
    class Meta:
        database = database
        only_save_dirty = True

    def __setattr__(self, name, value):
        if not hasattr(self, name) or getattr(self, name) is not value:
            return super().__setattr__(name, value)
```
Issue: Support for conditional insert/update based on EXISTS/NOT EXISTS
Author: dev-zero
State: closed
Created: 2016-02-12 11:39:32

I couldn't figure out how to do an "INSERT IF NOT EXISTS" according to http://stackoverflow.com/questions/15710162/conditional-insert-into-statement-in-postgres in PostgreSQL. I am now using `create_or_get` instead, but it would be nice if peewee would support the general "exists/not exists" in the `when` function.
Likewise for the `UPSERT` function introduced in PostgreSQL-9.5


Here's an example of how to insert with a select query:

http://docs.peewee-orm.com/en/latest/peewee/api.html#Model.insert_from

For postgres, you can use the on_conflict method:

http://docs.peewee-orm.com/en/latest/peewee/api.html#InsertQuery.on_conflict

I'll need to check if it works with postgres, as its only been tested on
sqlite.
On Feb 12, 2016 5:39 AM, "Tiziano Müller" notifications@github.com wrote:

> I couldn't figure out how to do an "INSERT IF NOT EXISTS" according to
> http://stackoverflow.com/questions/15710162/conditional-insert-into-statement-in-postgres
> in PostgreSQL. I am now using create_or_get instead, but it would be nice
> if peewee would support the general "exists/not exists" in the when
> function.
> Likewise for the UPSERT function introduced in PostgreSQL-9.5
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/coleifer/peewee/issues/853.


Ok, I tried the following:

``` python
models.TaskStatus.insert(name='done').on_conflict('IGNORE').execute()
```

which unfortunately generates a `INSERT OR IGNORE ...` which is not supported by PostgreSQL (I get a syntax error) instead of `sql INSERT ... ON CONFLICT` which is supposed to work with with PostgreSQL and MySQL.


Hi, any word on `upsert` support for Postgres 9.5?


I'm planning support for Postgres 9.5's new upsert functionality. Discussion has been consolidated into this ticket: #1307 . Please feel free to comment on that ticket with any thoughts or suggestions.
Issue: old SQLite3 has no sqlite3_compileoption_used function
Author: gilchris
State: closed
Created: 2016-02-12 10:09:49

If installed SQLite3 version is less than 2.6.23, [BerkeleyDatabase.check_libsqlite() function in berkeleydb.py](https://github.com/coleifer/peewee/blob/master/playhouse/berkeleydb.py#L102) raise AttributeError.

sqlite3_compileoption_used function in SQlite is added on 2.6.23. [link](https://www.sqlite.org/releaselog/3_6_23.html)


That's one old ass sqlite.
Issue: Postgresql Foreign key Migrations fail to rename columns
Author: Hendler
State: closed
Created: 2016-02-11 22:54:17

First, thanks for peewee. 

I've tried to include relevant bits for this example. Maybe I'm doing something wrong.

```
import basemodel
from peewee import CharField, DateTimeField, BigIntegerField, BooleanField, IntegerField, ForeignKeyField, Proxy, TextField, PrimaryKeyField
from playhouse.postgres_ext import BinaryJSONField, ArrayField
from playhouse.migrate import PostgresqlMigrator
from playhouse.migrate import migrate as mm
mdb = basemodel.dbfactory()
migrator = PostgresqlMigrator(mdb)

def migration_wrapper(*args):
    with mdb.transaction():
        mm(*args)


    migration_wrapper(
        migrator.add_column('blogpost', 'user', ForeignKeyField(User, related_name='posts', index=True, null=True, to_field=User.id))
)

```

This creates a column called `user`. The model file also has this field, as "user". BUT using create_table would create a column named `user_id`.
So, future inserts fail

```
post = BlogPost()
post.user = User.get(id=user_id)
post.save() # <- column not found

```

probably unrelated to #474 ?


Peewee uses the convention that foreign key columns end with "_id". No bug. Either change your migration to use user_id as the column, or in your field on the model specify db_column='user'


Thanks I got confused by the create_table and column specific operations.
Issue: ALTER_COLUMN with MariaDB
Author: ptbrowne
State: closed
Created: 2016-02-11 09:08:32

Hi,

I am using MariaDB (10.1.10 released on 2015-12.24) and peewee 2.8.0 and I am having trouble with a migration.

``` python
no_newsletter = BooleanField(default=False)
migrator.add_column(
     'user', 'user_no_newsletter', no_newsletter),
)
```

The SQL issued is:

```
('ALTER TABLE `user` ADD COLUMN `user_no_newsletter` BOOL', [])
('UPDATE `user` SET `user_no_newsletter` = %s', [False])
('ALTER TABLE `user` ALTER COLUMN `user_no_newsletter` SET NOT NULL', [])
```

And I get this error for the last query:

```
peewee.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'NOT NULL' at line 1")
```

I had success by manually changing the request to

```
('ALTER TABLE `user` MODIFY `user_no_newsletter` BOOLEAN NOT NULL', [])
```

Relevant MariaDB docs: https://mariadb.com/kb/en/mariadb/alter-table/

It seems that in MariaDB, ALTER COLUMN cannot be used to enforce NOT NULL on a column.

I am no expert nor in peewee nor in MariaDB so it may be me that is making a mistake somewhere.

Thanks for peewee !


That's odd, are you using the `MySQLMigrator`? The `add_not_null()` method appears to be using the "MODIFY" syntax.


My bad... That was it, I was using PostgreSQLMigrator >< Thanks
Issue: Subquery join replaces primary key
Author: purpurpur
State: closed
Created: 2016-02-10 10:54:59

Hello,

I have models
Category:
  uid - primary key
Tag:
  uid - primary key
CategoryTag:
  uid - promary key
  category - foreign key to Category
  tag - foreign key to Tag

And using this to get all categories with tag uid list for each category:

``` python
    categories_ = models.Category\
        .select()\
        .paginate(CATEGORIES_COUNT/LIMIT, paginate_by=LIMIT)\
        .alias('categories_')
    categories = models.Category\
        .select(models.Category, models.CategoryTag)\
        .join(
            categories_,
            on=(models.Category.uid == categories_.c.uid))\
        .join(models.CategoryTag, join_type=peewee.JOIN.LEFT_OUTER)\
        .aggregate_rows()
```

But, after that if I try to get uid (categories[0].uid) for any category it will be an empty list, but not uid value, if it is getting replaced, how can I fix this? Without that joining subqueries is useless


@embali you should use [code formatting](https://guides.github.com/features/mastering-markdown/#GitHub-flavored-markdown) to make your issue more readable.


Indeed, I'm confused why people expect me to spend time answering the question they were too lazy to write properly.


Try changing the join on the subquery to:

``` python
        .join(
            categories_,
            on=(models.Category.uid == categories_.c.uid).alias('xx'))\
```
Issue: Database name necessary?
Author: startakovsky
State: closed
Created: 2016-02-08 23:50:09

Maybe this is a feature improvement, but why does peewee require the database name in the url?

mysql://{username}:{password}@{host}:{port}  gives an error for something like this only in peewee, not sqlalchemy.


Peewee needs to know which database to connect to, so yes, this is a requirement.
Issue: Proposed fix for #845
Author: cledoux
State: closed
Created: 2016-02-05 17:53:57

APSWDatabase _connect() without also making a call to _add_conn_hooks().

This was resulting in the pragmas keyword for **init** being ignored as the call to _set_pragmas  was never made.

I've added test cases that illustrate my original problem and added the call to _add_conn_hooks() to APSWDatabase._connect().


Merged a slightly different fix. Thanks for bringing this to my attention.
Issue: APSWDatabase not respecting pragmas keyword.
Author: cledoux
State: closed
Created: 2016-02-05 17:49:03

When initializing an apsw database with pragmas, these pragmas seem completly ignored. This problem is not seen in the parent class, SqliteExtDatabase.

```
>>> db = APSWDatabase('/tmp/apsw.sqlite', pragmas=(('journal_mode', 'WAL'),))
>>> db.connect()
>>> db.execute_sql('PRAGMA journal_mode;').fetchone()[0]
'delete'
>>> db = SqliteExtDatabase('/tmp/sqliteext.sqlite', pragmas=(('journal_mode', 'WAL'),))
>>> db.connect()
>>> db.execute_sql('PRAGMA journal_mode;').fetchone()[0]
'wal'
```


I've created a pull request with test cases that illustrate the issue along with a potential fix.

APSWDatabase overwrites _connect without calling _add_conn_hooks. I added this missing call.


Thanks for the amazingly fast turnaround!
Issue: Allow to make dynamic related name with current class
Author: yohanboniface
State: closed
Created: 2016-02-05 17:26:46

When declaring a ForeignKey in an abstract class, this allows
to customize the related_name for each child.

For example:

``` python
class Other:
    pass

class Abstract:
    # Will create Other.child1_foo and Other.child2_foo accessors.
    other = ForeignKey(Other, related_name='{classname}_foo')

class Child1(Abstract):
    pass

class Child2(Abstract):
    pass
```

Thanks :)


What you have there is really no different from the current behavior. For instance:

``` python
In [1]: from peewee import *

In [2]: class User(Model):
   ...:     username = CharField()
   ...:     

In [3]: class BaseTweet(Model):
   ...:     user = ForeignKeyField(User)
   ...:     

In [4]: class Tweet(BaseTweet):
   ...:     pass
   ...: 

In [5]: class PhotoTweet(BaseTweet):
   ...:     pass
   ...: 
In [6]: User.phototweet_set
Out[6]: <peewee.ReverseRelationDescriptor at 0x7f42ac136710>

In [7]: User.tweet_set
Out[7]: <peewee.ReverseRelationDescriptor at 0x7f42ac136490>
```


> What you have there is really no different from the current behavior.

Well, I know that. But I want to take control over this naming…


@coleifer can you reconsider this? In my point of view it does not hurt, while allowing user to customize easily related names.
Issue: Support for `flat` tuples
Author: alexanderad
State: closed
Created: 2016-02-03 13:18:30

While working with peewee we often query database for a bunch of records on a single field and it is handy in this case to have a flat, chained resultset (functionality similar to [values_list(flat=True)](https://docs.djangoproject.com/en/1.9/ref/models/querysets/#values-list))

So instead of having

``` python
>>> User.select(User.id).tuples()
[(1,), (2, ), (3, )]
```

I imagine this could be expressed in API like:

``` python
>>> User.select(User.id).tuples(flat=True)
[1, 2, 3, 4, ...]
```

Can we consider having this on board, @coleifer? I'd work on corresponding PR if this fits the design.

/cc @stas


You can use `more_itertools` to flatten that.

``` python
In [1]: import more_itertools

In [2]: l = [(1,), (2, ), (3, )]

In [3]: list(more_itertools.flatten(l))
Out[3]: [1, 2, 3]
```


I can use `itertools` as well (which is part of stdlib contrary to `more_itertools`), but this is not a question _how_ to do that:

``` python
In [4]: list(itertools.chain(*[(1,), (2, ), (3,)]))
Out[4]: [1, 2, 3]
```

This is rather a convenient shortcut.


I'm going to pass on this feature, as it is fairly easy to implement on your own. If you really want, instead of a list of row tuples, a list of scalar values - then you can write either `map(query, operator.itemgetter(0))` or something similar.


+1, programmers are lazy
Issue: Support for MySQL STRAIGHT_JOIN
Author: fmazan
State: closed
Created: 2016-02-02 11:18:20

Would it be possible to somehow perform STRAIGHT_JOIN on MySQL tables due to its poor query optimizer except using raw SQL query?

The syntax according to [http://dev.mysql.com/doc/refman/5.7/en/join.html](http://dev.mysql.com/doc/refman/5.7/en/join.html) is:

> join_table:
>     table_reference [INNER | CROSS] JOIN table_factor [join_condition]
>   | table_reference STRAIGHT_JOIN table_factor
>   | table_reference STRAIGHT_JOIN table_factor ON conditional_expr
>   | table_reference {LEFT|RIGHT} [OUTER] JOIN table_reference join_condition
>   | table_reference NATURAL [{LEFT|RIGHT} [OUTER]] JOIN table_factor


Just do this:

``` python
from peewee import *
JOIN['STRAIGHT'] = 'STRAIGHT'

my_query = SomeObj.select().join(SomeOther, JOIN.STRAIGHT)
```


This doesn't work anymore in version 3, due to:


```
class Join(BaseTable):
...
    def __sql__(self, ctx):
        (ctx
         .sql(self.lhs)
         .literal(' %s JOIN ' % self.join_type)
         .sql(self.rhs))
        if self._on is not None:
            ctx.literal(' ON ').sql(self._on)
        return ctx
```

Are you sure about that? Seems to work for me:

```python

In [1]: from peewee import *

In [2]: JOIN['STRAIGHT'] = 'STRAIGHT'

In [3]: class Foo(Model):
   ...:     pass
   ...: 

In [4]: class Bar(Model):
   ...:     foo = ForeignKeyField(Foo)
   ...:     

In [5]: query = Bar.select().join(Foo, JOIN.STRAIGHT)

In [6]: query.sql()
Out[6]: 
('SELECT "t1"."id", "t1"."foo_id" FROM "bar" AS "t1" STRAIGHT JOIN "foo" AS "t2" ON ("t1"."foo_id" = "t2"."id")',
 [])

```

The problem is that the STRAIGHT_JOIN is written with an underscore, it's not space separated. Oh well, those MySQL quirks...

Ahh, thanks for the explanation. Let me look into it.

Yeah, the code doesn't seem to support that very well...going to leave as wontfix for now.