akumoth/peewee-issues huggingface dataset:
Issue: Float gets autoconverted into integer in query
Author: Budovi
State: closed
Created: 2016-12-16 22:21:20

The following code:
```python
import peeweee

db = peewee.SqliteDatabase(None)
class Table(peewee.Model):
    id = peewee.PrimaryKeyField()
    foo = peewee.IntegerField()
    bar = peewee.IntegerField()
    class Meta:
        database = db

db.init("test.db")
db.connect()
db.create_tables([Table])

Table.create(foo=10, bar=3)
Table.select().where(Table.foo * 0.5 > Table.bar).get()
```
Raises the following exception:
```python
Traceback (most recent call last):
  File "/home/budovi/.local/lib/python3.5/site-packages/peewee.py", line 3097, in get
    return next(clone.execute())
  File "/home/budovi/.local/lib/python3.5/site-packages/peewee.py", line 2271, in next
    obj = self.iterate()
  File "/home/budovi/.local/lib/python3.5/site-packages/peewee.py", line 2253, in iterate
    raise StopIteration
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/budovi/.local/lib/python3.5/site-packages/peewee.py", line 3101, in get
    % self.sql())
__main__.TableDoesNotExist: Instance matching query does not exist:
SQL: SELECT "t1"."id", "t1"."foo", "t1"."bar" FROM "table" AS t1 WHERE (("t1"."foo" * ?) > "t1"."bar")
PARAMS: [0]
```
Note the parameters passed to the query. The float got converted to integer.

This is inconsistent with shell behaviour:
```
$ sqlite3 test.db
sqlite> SELECT * FROM `Table` WHERE `foo` * 0.5 > `bar`;
1|10|3
```
and comparison / multiplication is defined with numeric types just fine (https://www.sqlite.org/datatype3.html).

Nice catch.

There have been issues raised in the past due to the opposite being the case -- Peewee was *not* converting the value on the right-hand-side to the appropriate data-type. The fix is pretty simple:

```python
from peewee import Passthrough

Table.select().where(Table.foo * Passthrough(0.5) > Table.bar).get()
```
Issue: MySQLDatabase object has no attribute '_meta'
Author: ambross
State: closed
Created: 2016-12-16 05:32:39

This is my simple code
```
from peewee import *

db = MySQLDatabase('peeweetest',host='localhost',port=3306,user='root',passwd='****')

class Hello(db):
	name = CharField()
	age = CharField()

	class Meta:
		database = db

db.connect()
db.create_table(Hello,safe=True)
```

When I run this I get 
```
Traceback (most recent call last):
  File "tm.py", line 13, in <module>
    db.create_table(Hello,safe=True)
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 3769, in create_table
    return self.execute_sql(*qc.create_table(model_class, safe))
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 2084, in inner
    return self.parse_node(fn(*args, **kwargs))
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 2103, in _create_table
    meta = model_class._meta
AttributeError: 'MySQLDatabase' object has no attribute '_meta'
```
peewee version 2.8.5

Hello should subclass Model, not your database instance.
Issue: Schema migrations don't create indexes
Author: alexlatchford
State: closed
Created: 2016-12-15 22:27:49

If you're doing a migration and add a column with `index=True` or `unique=True` it doesn't automatically create an index. Even trying to perform a `migrator.add_index()` in the same `migrate()` block fails.

I'm doing this with the Postgres backend.

The code to add an index is tested, so I believe adding an index explicitly should work. You can check the `test_add_index` test-case in `test_migrate.py`.

Call Model.create_table(). 

Or, oddly, database.create_tables([Model]). The database.create_table method will only create the table, though, whereas create_tables() creates both tables and indexes.

I've tested this with v2.8.6 and definitely fixed for the schema migration utils

@coleifer Yes sorry I removed my comment without updating.

`database.create_tables([])` do create both table and indexes but `database.create_table` does not.
Issue: Where is ClosureTable?
Author: Naatan
State: closed
Created: 2016-12-14 21:03:52

See docs:

http://docs.peewee-orm.com/en/latest/peewee/playhouse.html?highlight=apsw#ClosureTable

ClosureTable is not defined. Using it produces an "is not defined" error.

I found the problem, seems ClosureTable does not get defined when using apsw_ext. I'll close this and refile if I find I can't use it from sqlite_ext.
Issue: Documentation on Extensions?
Author: Naatan
State: closed
Created: 2016-12-12 16:42:24

I'm interested in SQLITE FTS4, FTS5 and closure tables. Peewee [states to have some form of support for them](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html?highlight=closure#sqlite-ext). However, that's the only mention I see. Are there no docs on what this support actually entails and how I use it? How would I go about taking advantage of Peewee's "support" for these features?

Nevermind, it's there. The header style was throwing me off.
Issue: Misunderstanding somethng or weird behaviour ?
Author: stuaxo
State: closed
Created: 2016-12-08 10:49:18

I've got a connection to a sqlite db and a table like this one :
```
class Resource(Model):
    # [snipped other fields]
    filename = CharField(null=True)
```

If I do 

```
resources = models.Resource.select().where(models.Resource.filename == None)
for r in resources:
    r.update(filename=filename)
    import ipdb; ipdb.set_trace()
```

Then r.filename is None...
```
>>> r.filename == None
True
```

Is this expected?  How can I set this value ?

Worse - if I use

```
r.filename = filename
r.save()
```
Data in other tables seem to get saved, with default values overwriting the proper values.

(I switched from save() because of this initial problem)... now I've tested going back, it has this other bug.

Hi,
   I'm going to close this.    - Project I'm on is on a silly deadline, so don't have time to simply reproduce it unfortunately.    Our app is has quite a few threads using asyncio, which may be the issue.  ..  It'll take about the same time to move our app to SqlAlchemy as it would to make a simple example, so were going on this path.

Cheers
S

You need to call ".execute()" to actually execute the update query.

Sorry I didn't see this sooner, but hopefully this information finds you in time to make your project work.

Cheers for the info :)
Issue: About connection pool
Author: 521xueweihan
State: closed
Created: 2016-12-08 05:35:09

Hi coleifer, 

First peewee is awesome, It is very useful. Thank you.

There I consult for something about `connection pool`. when I use the peewee connection pool, I meet some problem: `Exceeded maximum connections.` and `Error 2006: “MySQL server has gone away”`, so I read document and peewee source code solve this two Error. 

The reason is: I don't close the connection when I use the pool. So I try to find a way to fix it out.

The document give two way to auto close connection:

```python
with db.execution_context():
    # A new connection will be opened or, if using a connection pool,
    # pulled from the pool of available connections. Additionally, a
    # transaction will be started.
        for user in get_all_user():
            user_id = user['id']
            rule = Rule(user_id)
            rule_dict = rule.slack_rule()
            .....do something.....
```

```python
@app.before_request
def _db_connect():
    database.connect()
#
# This hook ensures that the connection is closed when we've finished
# processing the request.
@app.teardown_request
def _db_close(exc):
    if not database.is_closed():
        database.close()
#
#
# 
from playhouse.flask_utils import FlaskDB
from dock_fastgear.model.base import db
#
app = Flask(__name__)
FlaskDB(app, db) 
```

At last , I have a question: Why connection alway close by myself(Implicit or Dominant), Because can't check `_in_use` connections timeout ? There is no automatic management of connections, The feature will be there?

Connections must be closed by the application writer. It is a good practice to ensure that your connections are opened and closed in a sensible way.
Issue: No PasswordField in playhouse.fields
Author: Morriaty-The-Murderer
State: closed
Created: 2016-12-05 02:49:42

Hi

I got an error when trying to import PasswordField. Then I checked this
```python
>>> from playhouse import fields
>>> fields.P
fields.PY2                  fields.PickledField(        fields.PrimaryKeyField(     fields.Proxy(
fields.Param(               fields.PostgresqlDatabase(  fields.ProgrammingError(    
>>> import peewee
>>> peewee.__version__
peewee.__version__
>>> peewee.__version__
'2.8.5'
```

The only reason I supposed was that the os kernel was too old
```bash
$ uname -a
Linux prod-bj-sem-20-143 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
```
So should I install some extra libraries?

Sorry for ignoring this tip
>Note
>This field requires bcrypt, which can be installed by running pip install bcrypt.
Issue: Model.get_or_none
Author: razodactyl
State: closed
Created: 2016-12-04 12:46:09

I think the only place I found this behaviour was in the Dataset utility.

Usually, to get a row you would use Model.get.
However, if the row doesn't exist, a DoesNotExist is thrown.

I propose a 'get_or_none' function to return None instead of throwing an error, in this way, you can simply query a record and have nothing if there's no records.

e.g

```
user = User.get(username == 'razodactyl') # returns User
user = User.get(username == 'not_created_yet') # returns None
```

I'm currently using this on my 'BaseModel' implementation which is a subclass of 'db.Model' which came from the FlaskDB helper.

```
class BaseModel(db.Model):
    @classmethod
    def get_or_none(cls, *query, **kwargs):
        try:
            return cls.get(*query)
        except cls.DoesNotExist:
            return None
```

    user = User.select().where(username == 'not-created').first()

That's what I usually do. `first()` will return a record or None.

Thanks @timster
Issue: peewee connection pooling returns broken connections from the pool
Author: c0ldcalamari
State: closed
Created: 2016-12-02 19:46:06

If you execute some bad SQL like `select * from not_a_table;` postgres will leave you in a [CONNECTION_BAD](https://www.postgresql.org/docs/9.1/static/libpq-status.html) state.  Requiring you to rollback or reset the connection before continuing.  Peewee intentionally leaves this up to the user to handle (either through the [`autorollback`](https://github.com/coleifer/peewee/issues/240) param or otherwise).  If the user doesn't handle it, all future SQL statements will fail.

This is fine, but if the user *doesn't* handle it, the connection should not be put back into the pool as-is.  Connection pools should return functionally new connections each time.  The next user of the pool should not have to worry about what the previous user did or did not do.

Example of the pool returning a borked connection:

```python
import playhouse.db_url
DB_URL = 'postgresext+pool:///template1'
db = playhouse.db_url.connect(DB_URL)

db.connect()
try:
  db.execute_sql('select * from not_a_table;')
except Exception as e:
  print 'got', repr(e.message), 'but ignoring'

# close the bad connection, returning it to the pool
db.close()

# get a "new" connection from the pool
db.connect()
# this should not fail, but does
db.execute_sql('select 1;')
db.close()
```

`psycopg2` knows in-memory (not requiring a network call) the state of the connection, so it should be easy to detect without a performance hit, and offers a `reset()` method that fixes it, so this should be an easy fix.


Merged 082f13bf6b3a740c07bf35dafaf699e236b097e1
Issue: fix bad pg connections before returning to pool
Author: c0ldcalamari
State: closed
Created: 2016-12-02 19:44:20

For https://github.com/coleifer/peewee/issues/1138

Thanks!

This is fantastic, thank you. I will merge, but I'd like to tighten up the API first. Thanks again.

I've made some small cleanups/changes to the APIs but this feature is now merged. Thanks again.

082f13bf6b3a740c07bf35dafaf699e236b097e1
Issue: Remove `*` imports in doc
Author: maxnoe
State: closed
Created: 2016-12-02 17:17:30

The one thing that often tends to annoy me when reading the docs is the use of `from peewee import *` or `from playhouse.migrate import *`.

One never really knows where stuff comes from.

Yeah, I hear that. I guess, as the author, I'm always aware of that stuff but I can see how it would be super confusing in the docs. I'll update them as I find the time.

Well, I looked into this a bit. The `import *` are actually necessary in some places because extension modules typically will shadow peewee globals. Peewee has always been made with an eye towards `from peewee import *` so ...think I'm going to just leave this one alone.
Issue: Possibility to exclude certain columns from default selection
Author: maxnoe
State: closed
Created: 2016-11-30 20:08:54

The default for `peewees` `select()` is to request all columns.

Could it be possible to exclude columns from this selection?

E.g. something like:

```python

class MyModel(Model):
    x = FloatField()
    data = BlobField()

    class Meta:
        database = database
        exclude_from_default_select = [data]
```

I ran into some performance issues, because I somewhere always requested a large blob through a foreign key traversion.

Now I do something like

```python
b = ModelB.select(ModelB.some_field).where(ModelB.id == a.model_b_id).get()
some_field = b.some_field
```

Instead of just accessing
```
a.model_b.some_field
```
because `ModelB` has a large Binary blob.

It would really simplify my code if I could exclude the blob field from the foreign key traversion.

Thanks for the great package and for any help!

Just write a custom method.
Issue: DecimalField field does not attempt to coerce/convert values
Author: talawahtech
State: closed
Created: 2016-11-18 06:59:46

I have noticed that unlike most other numeric fields, DecimalField does not attempt to coerce/convert values (e.g. strings) before inserting them into the database.  

Is there a specific reason for this or is it just an oversight. Seems like it would to be consistent with the behavior of IntegerField and FloatField

If `auto_round` is set to `True` then peewee will call `str()` on the value before passing it into the `Decimal` constructor. Otherwise, peewee passes the value back to the database as-is.

On the other side, when loading values from the db into python, Peewee will convert the value to a string then pass it in to the Decimal constructor (if the value is not already a Decimal object).

Can you provide the specific example you're having trouble with?


Yes, I see that the conversion happens when auto_round is set, but I was expecting it to also happen without auto_round. For example, given the following string input: "1" IntegerField, FloatField, etc will all coerce the value to their corresponding numeric representations, while DecimalField will send the string along to the DB/driver as is.

On the flip side, given the string input "garbage223", using IntegerField will throw an Exception while a DecimalField will send the value to the DB. I discovered this issue when I realized that bad data in a DecimalField was silently converted to 0.0000 by MySQL and a pymysql warning was printed to the log instead. It seems much more consistent to attempt the conversion in peewee just like with the other numeric fields so that users can handle all exceptions in the same way.

Basically instead of simply returning value, I would expect you to return decimal.Decimal(str(value))


Actually, let me add that you should probably also check is_infinite() and is_nan() on the converted decimal values before returning or calling quantize().  Otherwise the strings "inf" and "nan" could cause trouble as well 


> when I realized that bad data in a DecimalField was silently converted to 0.0000 by MySQL

Shit! That's certainly surprising. I think it's ok to blame the problem on MySQL rather than Peewee...or pymysql...something should have broken higher up the call-stack.

That being said, I think it makes sense to add logic to convert to Decimal object in Peewee's db_value() method, so I've gone ahead and done that. Also skip quantization when the value is something like NaN/Inf, etc.
Issue: Max recursion depth exceeded - JSON Type query w/ Postgresql
Author: wmturner
State: closed
Created: 2016-11-18 06:46:25

Hello,  

I'm just getting started with peewee and am using Postgresql with a few JSON type columns.  When trying to query on one of the elements in the JSON document I get a maximum recursion level exceeded error. 

The code I have so far is here: https://gist.github.com/wmturner/7288bbf715d81df916cb6b561057e490

And the error looks like:

 File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 1823, in parse_node
    csql, cparams = self.parse_node(param)
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 1823, in parse_node
    csql, cparams = self.parse_node(param)
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 1823, in parse_node
    csql, cparams = self.parse_node(param)
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 1823, in parse_node
    csql, cparams = self.parse_node(param)
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 1806, in parse_node
    sql, params, unknown = self._parse(node, alias_map, conv)
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 1792, in _parse
    elif (isclass(node) and issubclass(node, Model)) or \
  File "/usr/lib/python2.7/inspect.py", line 65, in isclass
    return isinstance(object, (type, types.ClassType))
RuntimeError: maximum recursion depth exceeded while calling a Python object

I'm probably just doing it wrong or not seeing a glaring error, so any help would be greatly appreciated!

Best,
-William

The problem appears to be that you're using `PostgresqlDatabase` as opposed to `PostgresqlExtDatabase`. The former does not know how to parse JSON things into SQL, while the latter does. I admit the error message could be more helpful, but that's the issue. If you switch your DB initialization you'll be set.


Sweet, thanks for the help.
Issue: Math operators in query
Author: ak4nv
State: closed
Created: 2016-11-16 13:32:19

Hi!
I have a `peewee.DateTimeField`in my model. And I need calculate duration between date in this field and current date in days.
```python
from peewee import DatetTimeField, fn
from playhouse.shortcuts import cast
from config import db
from datetime import datetime

class Table(db.Model):
    ts = DateTimeField(default=datetime.now)


duration = fn.current_date - cast(Table.ts, 'date')
qs = Table.select(duration.alias('duration')).dicts()
```
But it not works. Where I have mistaken?
SQL query works fine:
```sql
select current_date - cast(ts as 'date') as duration from table
```


What errors do you receive? 
What SQL is composed?


Well, `fn.current_date` should be written as: `fn.current_date()`, unless that's some magical name in which case you should just write `SQL('current_date')`.

Math operators work as you'd expect, so theoretically your query would work provided your database understands the expression "current_date - cast(table.ts as date)".

Try fixing the function call as I described. It looks like you need to use the "SQL('current_date')" construction as opposed to the function call.


@coleifer great man!!! That's I wanted!
Issue: Not able to query in ForeignKeyField
Author: rahul3103
State: closed
Created: 2016-11-13 06:40:22

My `model.py`

```python
    from peewee import Model, CharField
    from playhouse.sqlite_ext import ForeignKeyField
    from playhouse.pool import SqliteExtDatabase
    
    import settings
    
    db = SqliteExtDatabase(settings.DB_NAME)
    
    
    class BaseModel(Model):
        """A base model that will use our Sqlite database"""
        class Meta:
            database = db
    
    
    class Restaurant(BaseModel):
        """ORM model of the Restaurant table"""
        name = CharField(null=False)
    
    
    class MenuItem(BaseModel):
        """ORM model of the MenuItem table"""
        name = CharField(null=False)
        course = CharField()
        description = CharField()
        price = CharField()
        restaurant = ForeignKeyField(Restaurant, null=False, db_column='restaurant_id')

```

When I am trying to query like this I am getting following error:

```
In [5]: menu = MenuItem.select().where(MenuItem.restaurant.id == 2).first()
    ---------------------------------------------------------------------------
    AttributeError                            Traceback (most recent call last)
    <ipython-input-5-4a02ff7da8b8> in <module>()
    ----> 1 menu = MenuItem.select().where(MenuItem.restaurant.id == 2).first()
    
    AttributeError: 'ForeignKeyField' object has no attribute 'id'
```

But I am able to query like this:

```
    In [7]: menu = MenuItem.select().first()
    
    In [8]: menu.restaurant.id
    Out[8]: 1
    
    In [9]: menu.restaurant.name
    Out[9]: u'Pizza Palace'
```

Same problem I am getting while:
```
menu = MenuItem.select().where(MenuItem.restaurant.name == 'Veggie Burger').first()
```
Here I am not getting error but I am getting zero output from the query even though I have rows for that query in db.

You're close, you would write it as:

``` python
MenuItem.select().join(Restaurant).where(Restaurant.name == name)

MenuItem.select().where(MenuItem.restaurant == restaurant_id)
```


Thanks
Issue: Extending PooledDatabase to optionally wait before throwing the "Exceeded maximum connections" exception
Author: decoratorz
State: closed
Created: 2016-11-11 16:25:57

We need to extend the Peewee's pool implementation with the timeout option which would allow to wait some time for connection when a pool is full instead of always throwing an exception immediately.  The aim is to do it "the right way" and have this change merged upstream. 

Problem is, it's hard to do locally in playhouse/pool because all calls to `_connect/_close` are protected with the `self._conn_lock` in the parent Database class and in ExecutionContext from the main module, so we can't wait inside `_connect` without manually releasing this upper level lock which doesn't look like a good solution.

Because this seemingly local feature could require refactoring in the main module we'd like to ask for your feedback first on how you see it implemented properly, or even see it in Peewee at all.

Thanks.

Well, theoretically you could implement this yourself by putting some kind of polling around the connect method. I think it'd require some re-architecting to get this working more generally. I'll think about it.


I've added this feature. You can now specify a `timeout` when instantiating a subclass of `PooledDatabase`. To block forever simply set the timeout to zero.