akumoth/peewee-issues huggingface dataset:
Issue: The update of BooleanField coerce
Author: mobishift2011
State: closed
Created: 2016-07-19 05:35:02

Good afternoon, coleifer. We've found the most common case is that the data from the client(as browser) will send true or false, 1 or 0 to the server, but the conversion should be decided by peewee on the data field type. So may be the update of mine makes common sense.


Is this in reference to a particular issue?


**Amazing！**


This patch seems more suitable for a subclass of BooleanField that is designed to handle things like strings or untrusted data.
Issue: Pass params into where with kwargs.
Author: alexpantyukhin
State: closed
Created: 2016-07-18 17:16:45

Hi. Sometimes it's very useful to pass a collection of parameters with `kwargs` into the `where` clause.
What if to add some kind of function like `q(class, **kwargs)` into playhouse which gets params and translate it on expressions which are already used in the peewee.
(Not sure if such function already exists). Or maybe create another method like `where_by` likes it's done in SQLAlchemy.


Actually, this functionality does exist, but is currently marked as deprecated and no longer documented. You can view the docs [on the 2.0.2 build](http://docs.peewee-orm.com/en/2.0.2/peewee/querying.html#SelectQuery.filter), however.

If you use the alternative `filter()` API, you can use Django-style kwargs lookups:

``` python
# Works pretty much like you expect:
active = User.filter(active=True)
some_users = active.filter(username__in=('charlie', 'huey', 'zaizee'))

# You can do "OR" using `DQ`:
editors = User.filter(DQ(is_admin=True) | DQ(is_staff=True))
```
Issue: Minor: logging of queries after final string interpolation
Author: bcattle
State: closed
Created: 2016-07-17 23:07:33

Hey, I love this library. The [docs](http://docs.peewee-orm.com/en/latest/peewee/database.html#logging-queries) say you can turn on logging of queries with 

```
# Print all queries to stderr.
import logging
logger = logging.getLogger('peewee')
logger.setLevel(logging.DEBUG)
logger.addHandler(logging.StreamHandler())
```

This produces output such as the following:

```
('
          SELECT count(DISTINCT username)
            FROM get_rows_created_days_ago_with_lookback(NULL::stats_daily_active_users, %(days_prior)s, %(lookback_days)s)
        ', {'days_prior': 1, 'lookback_days': 7})
```

In other words, it's printing the output before the final string interpolation step. 

By accessing the psycopg2 `cursor.query` attribute, you are able to get the output _after_ that final string interpolation step, returning:

```
SELECT count(DISTINCT username)
    FROM get_rows_created_days_ago_with_lookback(NULL::stats_daily_active_users, 1, 7)
```

Would it be clearer if we change the logging code to use the psycopg2 `cursor.query` property? Thanks. 


It would be much better! If I recall correctly, you can also use psycopg2.mogrify to get the final query. Unfortunately pysqlite doesnt implement this:
- http://bugs.python.org/issue9506
- https://github.com/ghaering/pysqlite/issues/20#issuecomment-26258837

Looks like mysql-python doesn't implement this either.

So, because it's just postgres, I think I'll favor lowest common denominator here.
Issue: Execute a transaction when use Read Slaves
Author: huangzhw
State: closed
Created: 2016-07-14 08:33:03

When we are executing a transaction, we want the connection stay the same. But if we use Read Slaves, select will use slave connections? So, if we want use master connection, we have to instantiate the SelectQuery by hand?


For that case I'd suggest using the [execution context helper](http://docs.peewee-orm.com/en/latest/peewee/api.html#ExecutionContext).

Hard to say more, though. So you're using the "ReadSlaveModel"...may I ask what for? Do you have a large number of read replicas and a lot of database load?


No. I just want to figure out whether some operation done when execute a transaction. Maybe when in a transaction, the ReadSlaveModel should return the  master connection.


I'm not sure I follow, can you explain giving me a concrete example?


We did not user orm before. In our system, we have two mysql user,  db_reader and db_writer.  We use db_reader to execute read-only statement, and db_writer execute other statement.  When use peewee, I think the ReadSlaveModel can solve this situation.  


@hzweveryday -- thanks for the explanation of your database setup. Hmm... I know the `ReadSlaveModel` option sounds like the right solution, but I'm not sure how to go about ensuring the proper isolation between transactions/connections. This comes down to my lack of familiarity with the ins-and-outs of MySQL..

I'm not actually sure what benefits having two particular database users provides you, if your application is freely using both the reader and the writer. If they are both connected  to the same database, perhaps you could just use the db_writer exclusively?

I'd suggest that you go ahead and try using the `ReadSlaveModel` if you plan to maintain a strict separation between the two users/connections. Otherwise, I'd suggest using the connection pool with the db_writer user.


Thank you! Now I use PooledMySQLDatabase. This should be a good solution.
Issue: null FKs return non-null objects with all-null attributes
Author: neopunisher
State: closed
Created: 2016-07-13 17:45:00


Here's the plan for the fix:

> If the user selects the foreign key field and it is null, then it's a safe bet to skip object creation. Same, as you mentioned, if they select the related object's primary key. Beyond that it gets murky. I'll look into implementing this.

**Edit: fix has been merged**
Issue: Calling Model.save() changes PaswordField passwords
Author: joeyespo
State: closed
Created: 2016-07-13 03:10:42

I have a `User` model with a `PasswordField` and noticed that when I change some fields can call `user.save()`, the password gets regenerated. From skimming the code, it looks like it's re-hashing the password hash.

The following is my current workaround:

``` py
from playhouse.fields import PasswordField as PasswordFieldBase, PasswordHash


class PasswordField(PasswordFieldBase):
    def db_value(self, value):
        if isinstance(value, PasswordHash):
            return value
        return super(PasswordField, self).db_value(value)
```

If it matters, I'm using `flask_utils` and `Proxy` on `master` (since it contains [the proxy fix](https://github.com/coleifer/peewee/issues/933)).


That was _fast_! How do you do it? :smiley: 


Also, thanks!
Issue: Support for CASE or FILTER in COUNT
Author: dev-zero
State: closed
Created: 2016-07-12 13:14:31

Given a table `tas` with the following columns:

```
id | machine | status
```

I would like to count the number of total and running tasks per machine.

In PostgreSQL 9.5 this is possible using:

``` sql
SELECT machine,
  COUNT(id) FILTER (WHERE status='running') AS running,
  COUNT(id) AS total
  FROM task
  GROUP BY(machine);
```

So I naively write:

``` py
       q = Task.select(Task.machine,
                    fn.COUNT(Task.machine).alias('total'),
                    fn.COUNT(Task.machine).where(Task.status == "running").alias('running')) \
                .group_by(Task.machine)
```

which gives me something like

``` sql
SELECT "t1"."machine", COUNT("t1"."id") AS total, filter("t1"."status" = 'running') AS running FROM "task" AS t1 GROUP BY "t1"."machine"'
```

Unfortunately, PostgresSQL prior to 9.5 seems to need a slightly different construct:

``` sql
SELECT machine,
  COUNT(CASE WHEN status='running' THEN id END) AS running,
  COUNT(id) AS total
  FROM task
  GROUP BY(machine);
```

From reading the code I guess one would have to write a function similar to the `over()` already present in `Func`.


I came up with the following hacks/workarounds:

``` python
        q = Task.select(Task.machine,
                    fn.COUNT(Task.id).alias('total'),
                    fn.COUNT(SQL('t1.id) FILTER (WHERE t1.status = %s', 'running')).alias('running')) \
                .group_by(Task.machine)

```

``` python
       q = Task.select(Task.machine,
                    fn.COUNT(Task.machine).alias('total'),
                    fn.COUNT(SQL('CASE WHEN t1.status = %s THEN t1.id END', 'running')).alias('running')) \
                .group_by(Task.machine)
```


There is support for `Case`: http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#case

For other arbitrary ones, you can do:

``` python
Clause(
    fn.COUNT(Task.id),
    fn.FILTER(Clause(SQL('WHERE'), Task.status == 'running'))
)
```
Issue: [Bug] Prefetch and limit bug
Author: elkranio
State: closed
Created: 2016-07-08 17:46:28

Each game can have several platforms. When I use **limit** or **paginate**, some of the games end up without platforms. 

Acts strangely: 
`games = Game.select().limit(50)`
`platforms = GamePlatform.select()`
`games_pf = prefetch(games, platforms)`

Works fine
`games = Game.select().where(Game.id << [1, 2, 3 ... 50]`
`platforms = GamePlatform.select()`
`games_pf = prefetch(games, platforms)`

Not sure if it is a bug, but since your docs say

> Additionally, because of the semantics of subquerying, there may be some cases when prefetch does not act as you expect (for instance, when applying a LIMIT to subqueries, but there may be others) – please report anything you think is a bug to github.
Issue: peewee create index Error
Author: rfyiamcool
State: closed
Created: 2016-07-06 07:07:17

I want to create table with index, But raise error.    Mysql peewee version = 2.8.1

```
class ArticleContent(Model):
    article_id = IntegerField()
    content = BlobField()
    content_md5 = CharField(null=True,default=''")

    class Meta:
        database = db
        db_table= 'mirro_article_content'
        indexes = (
           (('article_id','content_md5'), False)
        )
```

Error log:

```
Traceback (most recent call last):
  File "core/models.py", line 138, in <module>
    init_table_data()
  File "core/models.py", line 133, in init_table_data
    ArticleContent.create_table()
  File "/Library/Python/2.7/site-packages/peewee.py", line 4562, in create_table
    cls._create_indexes()
  File "/Library/Python/2.7/site-packages/peewee.py", line 4586, in _create_indexes
    db.create_index(cls, fields, unique)
  File "/Library/Python/2.7/site-packages/peewee.py", line 3549, in create_index
    'or tuple: "%s"' % fields)
ValueError: Fields passed to "create_index" must be a list or tuple: "article_id"

```


You need a trailing comma after (('article_id', 'content_md5'), False) -- otherwise, since it's the only item in the outer-most parentheses, Python treats it as a normal expression rather than a tuple.
Issue: Added PyPy support
Author: thedrow
State: closed
Created: 2016-07-05 04:37:05


This will require a bit more work.
On PyPy we can use psycopg2-cffi.
Let me clone that branch. The build will pass soon.


I think pip is being silly here. Lemme check.


Now it skips the damm Cython generation step. I'll get back to you on that.


Closing for now, if you plan to re-open would you mind rebasing the existing commits into a single commit?


I wouldn't. It doesn't work yet though.


I mean I wouldn't mind rebasing...


Cool, I guess I should ask: is this something you plan on continuing to hack on until it's all working? If so be happy to reopen and you can rebase when it's closer to being done.
Issue: Added a pip cache to Travis
Author: thedrow
State: closed
Created: 2016-07-05 04:27:16


What's going on in this PR? I'm not sure I understand the differences here.


It caches the compilation of the C extensions and also doesn't download pip dependencies.
The build as a result should be faster.


Just looking at the build times it doesn't appear to have made a difference.


Did a quick skim of the travis cache docs, I'm going to pass on this one.
Issue: add "is_not", "not_in" django-style filter support
Author: qulc
State: closed
Created: 2016-07-03 08:52:15

peewee filter not support `isnull`
https://docs.djangoproject.com/en/1.9/ref/models/querysets/#isnull

so i want add 'is_not' mapping to `OP.NOT_IN`

``` python
Blog.filter(user__username__not_in=['u1', 'u2'], pub_date__is_not=None)
```


This is a nice PR, very clean, but the Django-style querying is deprecated and I do not plan on adding new functionality as it will be removed soon in the next major version...which hopefully isn't too far away!
Issue: Support cockroachdb?
Author: jet10000
State: closed
Created: 2016-06-28 01:02:22

https://www.cockroachlabs.com/blog/building-application-cockroachdb-sqlalchemy-2/


[From one of their devs](https://github.com/cockroachdb/cockroachdb-python/issues/21#issuecomment-229755195),

> Cockroach actually supports the Postgres wire protocol, so the Postgres driver builtin to Peewee should work (we support psycopg2 without issue). That said, we have found that a few drivers use some Postgres-specific SQL extensions (for instance, use of pg_catalog instead of the standard information_schema), so current levels of support for them vary.

This means that peewee, configured with a modified version of the `PostgresqlDatabase`, should be able to work with Cockroach -- at least in some sense.

Going to close this out as I do not plan on implementing support myself.

Those interested can also follow along here:

https://github.com/cockroachdb/cockroachdb-python/issues/21
Issue: db.create_table does not create indexes
Author: sin5th
State: closed
Created: 2016-06-27 10:41:40

class Client(peewee.Model):
    """
      UNIQUE KEY `rip_rport_stime` (`ip_type`, `remote_ip`, `remote_port`, `start_time`),
      KEY `app_stream_stime` (`app`, `stream`, `start_time`),
      KEY `app_stime` (`app`, `start_time`)
    """
    ip_type = UnsignedSmallIntegerField()
    app = UnsignedIntegerField()
    stream = UnsignedBigIntegerField()
    url = UnsignedIntegerField()
    start_time = UnsignedIntegerField()
    end_time = UnsignedIntegerField()
    remote_ip = UnsignedIntegerField()
    remote_port = UnsignedSmallIntegerField()
    local_ip = UnsignedIntegerField()
    local_port = UnsignedSmallIntegerField()

```
class Meta:
    database = db
    indexes = [
        (('ip_type', 'remote_ip', 'remote_port', 'start_time'), True),
        (('app', 'stream', 'start_time'), False),
        (('app', 'start_time'), False),
    ]
```

Client.create_table(fail_silently=True) # this create the right indexes
db.create_table(Client, safe=True) # but this can't

is it a bug? or i missed the doc?


Hey @sin5 - can you please take the 5 extra seconds and learn how to format your code? Did you even look at your issue after creating it?


Please read the documentation on submitting bugs: http://docs.peewee-orm.com/en/latest/peewee/contributing.html#bugs

Here's a link to GitHub's documentation:

https://help.github.com/articles/getting-started-with-writing-and-formatting-on-github/


Specifically, though, and this is a bad API -- I will be addressing it in a subsequent release -- calling `database.create_table` will _only_ create the table. Calling `Model.create_table()`, on the other hand, will create the table and the indexes.

However,

To create multiple tables, and their indexes, with the dependency graph resolved to ensure the tables are created in the proper order, use the `Database.create_tables()` method.

In fact, I pretty much just recommend that people **always** use `database.create_tables([AModel, AnotherModel])`.


Also read the big fat warning in the docs:

http://docs.peewee-orm.com/en/latest/peewee/api.html#Database.create_table


```
        sorry for my code format. and thanks for your answer：）
```

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or mute the thread.


No probs!
Issue: get tables commit transaction
Author: kaito-kidd
State: closed
Created: 2016-06-23 02:15:34

```
def get_tables(self, schema=None):
    return [row for row, in self.execute_sql('SHOW TABLES')]
```

execute `execute_sql` method auto commit transaction.

but is it necessary to do so?

If often call `create_table` method will lead to too many invalid commit.


How does this issue manifest itself?


For example, I have a log table, and is split by date table, every time you insert new data, checks the table exists, then call `create_table` methods, parameters `fail_silently = True`, it does not exist is automatically created to perform the `execute_sql` method.

the emergence of a large number of `show tables; commit;` 
`show tables` operation is not whether it is necessary to `commit`?


See #727, #806, #814 for why Peewee tends to issue commits with MySQL. In general I've found that users of the database are confused when it appears results are "cached" or they're not seeing changes. This happens when they have a long running process, for instance, and are issues reads with no commits.

If you would like to avoid the extra `COMMIT` statements, I'd suggest wrapping the code in a transaction using:

``` python
with db.atomic():
    # your code here.
```

Also, I don't know anything about your use case but it seems incredibly inefficient to be checking for the table existence before every insert statement when the data being inserted is log records. Just my .02 but that seems problematic.
Issue: Peewee throws error on cascade deletion if imports aren't present
Author: Coderdreams
State: closed
Created: 2016-06-21 13:45:39

I have something similar to this:

class Test(BaseModel):
    name = CharField()

class Test2(BaseModel):
    test = ForeignKeyField(Test)

class Test3(BaseModel):
    test2 = ForeignKeyField(Test2)

I had a script that said:

from database.model.test import Test
test = Test.get(Test.name == testName)
test.delete_instance(recursive=True, delete_nullable=True)

And it threw something along this:

_mysql_exceptions.IntegrityError: (1451, 'Cannot delete or update a parent row: a foreign key constraint fails (`mydatabase`.`test3`, CONSTRAINT `test3_ibfk_1` FOREIGN KEY (`test2_id`) REFERENCES `test2` (`id`))')

And it fixed itself only when I added the imports for the dependent models like this:

from database.model.test import Test
from database.model.test2 import Test2
from database.model.test3 import Test3
test = Test.get(Test.name == testName)
test.delete_instance(recursive=True, delete_nullable=True)

This poses the problem that some linters are going to complain about unused imports, I don't know if something can be done to improve this though. Also, I found out what the problem was only by chance, maybe capturing these exceptions and showing some hint to the users that the imports are needed could help someone else.

Thanks!


Please read the documentation on submitting bugs: http://docs.peewee-orm.com/en/latest/peewee/contributing.html#bugs

Specifically, take the extra 5 seconds and please format your code:

https://help.github.com/articles/getting-started-with-writing-and-formatting-on-github/
Issue: Fix truncate_date for PostgreSQL
Author: Zverik
State: closed
Created: 2016-06-21 12:38:23

The following script doesn't work:

``` python
#!/usr/bin/env python
import peewee, datetime

database = peewee.PostgresqlDatabase('mmwatch', host='localhost')

class Number(peewee.Model):
    date = peewee.DateTimeField(default=datetime.datetime.now)

    class Meta:
        database = database

database.connect()
database.create_tables([Number], safe=True)
number = Number()
number.save()
q = Number.select(database.truncate_date('day', Number.date))
for row in q:
    print row.date
```

It fails with the following error:

```
Traceback (most recent call last):
  File "./date_trunc_pgsql_fail.py", line 19, in <module>
    for row in q:
  File "/Users/ilyazverev/Progr/git/mmwatch/mmwatch/venv/lib/python2.7/site-packages/peewee.py", line 2967, in __iter__
    return iter(self.execute())
  File "/Users/ilyazverev/Progr/git/mmwatch/mmwatch/venv/lib/python2.7/site-packages/peewee.py", line 2960, in execute
    self._qr = ResultWrapper(model_class, self._execute(), query_meta)
  File "/Users/ilyazverev/Progr/git/mmwatch/mmwatch/venv/lib/python2.7/site-packages/peewee.py", line 2656, in _execute
    return self.database.execute_sql(sql, params, self.require_commit)
  File "/Users/ilyazverev/Progr/git/mmwatch/mmwatch/venv/lib/python2.7/site-packages/peewee.py", line 3492, in execute_sql
    self.commit()
  File "/Users/ilyazverev/Progr/git/mmwatch/mmwatch/venv/lib/python2.7/site-packages/peewee.py", line 3316, in __exit__
    reraise(new_type, new_type(*exc_args), traceback)
  File "/Users/ilyazverev/Progr/git/mmwatch/mmwatch/venv/lib/python2.7/site-packages/peewee.py", line 3485, in execute_sql
    cursor.execute(sql, params or ())
peewee.ProgrammingError: column "day" does not exist
LINE 1: SELECT DATE_TRUNC(day, "t1"."date") FROM "number" AS t1
                          ^
```

The reason is that `'day'` is passed to the query builder as-is, not escaped as a string. This pull request fixes that.


Cool, thanks!
Issue: bring select.get() into parity with model.get()
Author: keredson
State: closed
Created: 2016-06-17 15:16:57

currently this is allowed:

``` python
u = User.get(User.id==1)
```

and this is allowed

``` python
u = User.select().where(User.id==1).get()
```

but this is not:

``` python
u = User.select().get(User.id==1)
```

i suggest we impl the last option.  (ie make it identical to the middle option just like the first option is)


I would appreciate it if you respected my request that you not post things on my project.

You have your own fork, please use it.


i'm sorry, but you don't own github.  i'm allowed to discuss this project with other users/devs (just like anyone else).  you don't have to be involved in that discussion.


Derek, you seem to misunderstand free speech or your rights on GitHub.

While I might not agree with Charles, I definitely think it is his right to
ask you not to post here.
On Jun 17, 2016 7:41 PM, "Derek Anderson" notifications@github.com wrote:

> that is patently false. you've definitely deleted comments before issues
> were closed. and you've not deleted other's comments after issues were
> closed where you've deleted mine. (even when my comment was just "i agree
> w/ person XYZ"). i don't even know how many, as GH doesn't notify me when
> it happens, nor do i have any record of them (other than me luckily having
> left the browser open for that one screenshot)
> 
> besides, even the premise for your deletion is invalid. where is there a
> rule of "you can't comment after an issue is closed"? where else would one
> ever say "IMO this is still and issue and should be re-opened, because
> reason XYZ"?
> 
> refrain from commenting, creating tickets or posting here.
> 
> sorry, but no. if i have something to say that's respectful and on topic,
> i intend to reserve the right to say it. freedom of speech and whatnot.
> 
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> https://github.com/coleifer/peewee/issues/993#issuecomment-226819371,
> or mute the thread
> https://github.com/notifications/unsubscribe/AAEXLPEfY81fTY9jkYow2gQrxNrkJD2vks5qMs4rgaJpZM4I4daH
> .


What am I supposed to think, @keredson . You are a toxic presence on my project, and I don't want you to keep harassing me.


Why can't you respect that, one person to another?
Issue: djpeewee has no support for Field.db_column
Author: nschagen
State: closed
Created: 2016-06-17 12:46:19

Hi,

I would like to use PeeWee in an existing django project to run analytics queries. I tried it in the Django shell, but the queries failed because my model contained a field whose name did not match the actual database column name. This is because I am using `Field.db_column` for many fields. I'm using 2.8.1.

It would be great if the djpeewee `translate()` function would support models with custom `db_column` values.  Such behaviour could easily be enabled using an option passed to `translate()` to preserve backwards compatibility.

Thank you.