akumoth/peewee-issues huggingface dataset:
Issue: Create makes table, but no columns
Author: Neotric
State: closed
Created: 2016-03-21 15:18:27

Hello,
Tried my first steps with peewee, looks good, but....
I did a create_table(mytable), the table was created, but none of the defined columns in the class mytable. I want three columns, all CharField(). I only get an column "id" INTEGER NOT NULL PRIMARY KEY. On data insertion rows are created with one column ("id"), but no other data.
Who can help me ? What is going wrong ?


I am positive there is a bug in your code somewhere. I'd suggest posting on stackoverflow and INCLUDE YOUR MODEL DEFINITION and code you're using to create the table. That'll help get the problem fixed the fastest.

This page is for tracking bugs with peewee and creating tables is covered extensively by tests.
Issue: What is the reason to use keyword argument in positional style?
Author: kiddten
State: closed
Created: 2016-03-20 19:17:05

https://peewee.readthedocs.org/en/2.0.2/peewee/cookbook.html#using-with-postgresql

``` python
psql_db = PostgresqlDatabase('my_database', user='code')
```

Isn't 

``` python
psql_db = PostgresqlDatabase(database='my_database', user='code')
```

better?

I understand that that is matter of taste. But it's kind of official docs.


No, it's not better. It's just that you think it's better.
Issue: LookupError: unknown encoding: utf8mb4
Author: fraba
State: closed
Created: 2016-03-18 10:53:16

I'm not sure whether the problem is linked to my specific settings, but it seems I can't do `table_exists` on connection with a mysql database using `charset = utf8mb4`.

I previously altered my database with 

```
SET NAMES utf8mb4;
ALTER DATABASE my_database CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;
```

but 

```
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 4548, in table_exists
    return cls._meta.db_table in cls._meta.database.get_tables(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 3879, in get_tables
    return [row for row, in self.execute_sql('SHOW TABLES')]
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 3454, in execute_sql
    cursor.execute(sql, params or ())
  File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
    self.errorhandler(self, exc, value)
  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
LookupError: unknown encoding: utf8mb4
```


This error is coming from the MySQL driver so I suspect that peewee isn't the issue... But.

Does peewee need to run any special queries before the create table query?
Issue: does peewee support tormysql?
Author: indoorwise
State: closed
Created: 2016-03-14 07:55:45

examples of peewee with mysql use pymysql，but this library is synchronized，tormysql is asynchoronized library, peewee can support it?


I am not sure, but my guess is no. If you want some inspiration about implementing Peewee with event loops, check out https://github.com/05bit/peewee-async/blob/master/peewee_async.py
Issue: count() doesn't work properly using UNION
Author: tarot231
State: closed
Created: 2016-03-13 06:16:34

Hello,

following code currently doesn't work properly.

```
q = (q1 | q2)
print(q.count())
```

I tested to modify peewee.py as follows.

```
    def count(self, clear_limit=False):
        if self._distinct or self._group_by or self._limit or self._offset or \
                isinstance(self, CompoundSelect):
            return self.wrapped_count(clear_limit=clear_limit)
```

And then it went well.
Please confirm whether this is right.


Thank you, fixed.
Issue: model_to_dict with DateField
Author: alexpantyukhin
State: closed
Created: 2016-03-12 15:33:32

Hi!
I have such model:

```
class User(BaseModel, UserMixin):
    birth_date = DateField(null=True)
    ......
```

And when I try to do this:

`model_to_dict(current_user)`

I'm getting this error:
TypeError: datetime.date(1989, 8, 9) is not JSON serializable


But `model_to_dict` doesn't do any json serialization?


Sorry. found the reason it was another


alexpantyukhin , what was the reason? I am having the same problem.


model_to_dict returns a dict, not a JSON object, so I'm not sure what the confusion is about.


Sorry, model_to_dict is working as expected. I am making a confusion here.
I am trying to use json.dumps in a dictionary returned by model_to_dict, and it is failing (as expected).
The solution is not related to peewee code. 
Sorry for this unnecessary message!
Issue: SELECT FOUND_ROWS()
Author: daoiqi
State: closed
Created: 2016-03-11 13:00:49

``` sql
select * from t_table  where id > 10;
SELECT FOUND_ROWS();
```

peewee has found rows ?

need do it by self?

``` sql
select * from t_table  where id > 10 limit 10;
select count(0) from t_table  where id > 10;
```


use `wrapped_count()`

http://docs.peewee-orm.com/en/latest/peewee/api.html#SelectQuery.wrapped_count
Issue: Building complex queries programmatically.
Author: brijeshb42
State: closed
Created: 2016-03-10 11:05:04

This is not an issue but a question.
I have to build complex queries programmatically.

For ex:

For this model:

``` python
class Article(Model):
    title = CharField(null=False, unique=False, index=True)
    permalink = CharField(default=None, unique=True, null=True)
    tags = ArrayField(CharField, null=True, default=[], index=True)
    keywords = ArrayField(CharField, null=True, default=[], index=False)
    summary = TextField(null=True)
    content = TextField(default='')
    series_id = IntegerField(default=0, index=True)
```

Through an API request on web, I will receive the filter like this,

``` json
{
  "fields_wanted": ["title", "permalink", "summary"],
  "series_id": [1, 2]
}
```

Through the above post request data, I basically want to generate this query:

``` python
articles = Article.select(Article.title, Article.permalink, Article.summary).where(
   Article.series_id.in_(1, 2)).execute()
```

I was thinking to generate like this (after validating the json):

``` python
fields = [getattr(Article, field) for field in post_data["fields_wanted"]]
articles = Article.select(fields).where(
    getattr(Article, "series_id").in_(data["series_id"])).execute()
```

Is this a good way or can it be implemented in some other way?
How would I go about doing the above?


This type of question is not really appropriate for the peewee issue tracker.
Issue: Compound searches
Author: samizdis
State: closed
Created: 2016-03-10 02:49:39

```
select * from post t1
where not exists
(select 1 from tag t2
   where t1.id == t2.post_id and t2.name=='BadTag');
```

finds all posts which haven't any associated tags named 'BadTag'.

Is there a way to do this naturally in Peewee?

```
Post.select().where(
    ~Tag.select()
    .where(Post.id == Tag.post & Tag.name=='BadTag')
    .exists()
)
```

gets compiled to
`SELECT "t1"."id", ... FROM "post" AS t1 WHERE ? [-1]`
and
`Post.select().join(Tag).where(Tag.name!='BadTag')`
doesn't work since a Post can have many Tags.

I can use `Post.raw()` but then can't do things like `.order_by()`


One way to accomplish the "posts without badtag" might be:

``` python
(Post
 .select()
 .where(~fn.EXISTS(
      Tag.select().where(
          (Tag.post == Post.id) & (Tag.name == 'BadTag'))))
```

Another might be:

``` python
(Post
 .select(Post, fn.COUNT(Tag.id))
 .join(Tag, JOIN.LEFT_OUTER)
 .where(Tag.name == 'BadTag')
 .group_by(Post)
 .having(fn.COUNT(Tag.id) == 0))
```

I didn't test these examples, but hopefully you get the idea.
Issue: Key Error where query jsonb in postgresql with peewee
Author: bukun
State: closed
Created: 2016-03-09 14:18:40

I'm user peewee for PostgreSQL query with Python 3.5 under Debian Jessis. Everythin goes well untill querying Jsonb field.

The script:

```
import peewee
from playhouse.postgres_ext import BinaryJSONField

dbconnect = peewee.PostgresqlDatabase(
    'pycate',
    user='yunsuan',
    password='passhere',
    host='127.0.0.1',
    autocommit=True,
    autorollback=True
)

class BaseModel(peewee.Model):
    class Meta:
        database = dbconnect

class TestTabInfor(BaseModel):
    uid = peewee.CharField(null=False, index=True, unique=True, primary_key=True, max_length=36, help_text='', )
    title = peewee.CharField(null=False, default='')
    infor = BinaryJSONField()
try:
    TestTabInfor.create_table()
except:
    pass

condition = {}
db_data = (TestTabInfor
         .select()
         .where(TestTabInfor.infor.contains(condition)))
print(db_data)
```

It will echo:

`KeyError: 'JB@>'`


I have tried with Python 2.7 and Python 3.5.


Are you absolutely positive that your model class is using `PostgresqlExtDatabase` as it's database class?


Greate! thanks very much. I have tried several time, and could not find any information with google.
should be:

```
from playhouse.postgres_ext import PostgresqlExtDatabase
dbconnect = PostgresqlExtDatabase(
```
Issue: Handling of extra fields during create and insert
Author: poliquin
State: closed
Created: 2016-03-09 00:08:06

The `create` and `insert` methods of `Model` treat extra keys differently.  The following toy example demonstrates what I mean:

``` python
from peewee import Model, SqliteDatabase, IntegerField

db = SqliteDatabase(':memory:')


class ToyModel(Model):
    x = IntegerField()
    y = IntegerField()

    class Meta:
        database = db
        db_table = 'toys'


db.create_table(ToyModel, safe=True)

# insert some data, notice the extra 'z' key is ignored
data = {'x': 2, 'y': 4, 'z': 1}
m = ToyModel.create(**data)
m.save()  # no error here

# show that it worked
assert [(t.x, t.y) for t in ToyModel.select()] == [(2, 4)]

# insert some data, raises a KeyError for 'z'
ToyModel.insert(**data).execute()
```

`Model.insert` looks for each key in `cls._meta.fields`, but `create` and `__init__` simply iterate over the keywords and assign attributes for everything, regardless of whether there is an actual field in the model.

I prefer the behavior of ignoring keys without corresponding model fields (i.e. making `insert` consistent with `create`), but Python's `csv.DictWriter` is an example of allowing both behaviors via the `extrasaction` argument. Perhaps `peewee` could use a similar interface?

Is there a reason for `Model.create` to ignore "extra" keyword arguments and for `Model.insert` to throw an error? 


> Is there a reason for Model.create to ignore "extra" keyword arguments and for Model.insert to throw an error?

It's just an oversight on my part. At present, I am not inclined to change this as people may have code that relies on the behavior of `create()`. Additionally, `create()` is a bit semantically different than `insert()`....so I think I'm going to leave this as wontfix.


> I prefer the behavior of ignoring keys without corresponding model fields

Out of curiosity, @poliquin, what is your use case for that feature? I always thought I prefer the opposite and rather throw an error when trying to assign values to non-existent fields.


@tuukkamustonen, I find it useful when processing messy CSV data with fields that change slightly from file to file and have many uninteresting columns. I frequently encounter data like this from government sources. I read all data with `csv.DictReader`, transform fields to have consistent names matching my schema (e.g. lowercase, no spaces, ....), then simply pass the keyword arguments to `Model.create` with the `**` syntax. The stuff I want gets loaded, and the junk is ignored.

I definitely see the logic in throwing errors when encountering unexpected inputs, however, which is probably why the default `csv.DictWriter` behavior is `extrasaction='raise'`.
Issue: Missing implementation of __hash__ in Model
Author: kramer65
State: closed
Created: 2016-03-06 15:03:47

I'm using peewee in several of my projects, and I love it. Today I found a weird behaviour of testing whether models already exist in a dict. I asked a question about it on Stackoverflow: http://stackoverflow.com/questions/35826534/object-in-list-behaves-different-from-object-in-dict

[The answer](http://stackoverflow.com/a/35826785/1650012) turns out to be that the object (in this case an instance of a Model which I got from a peewee `select()`) doesn't seem to implement `__hash__` so it instead uses the “bad hash implementation” (which is based on the object id) as the default. I solved my trouble with al little workaround by using `if theObject in list(theDict)`, which makes use of the `__eq__` method instead of the `__hash__` method. 

I just wanted to let you guys know of this small issue though, so when you have time you can check whether you can add an implementation of the `__hash__` method to Model.

Cheers, and thanks for creating peewee!
Issue: limit(0) has no effect
Author: ameade
State: closed
Created: 2016-03-03 03:56:53

When doing for example 'User.select().limit(0)' all users will be returned. Changing the 0 to a positive integer seems to work correctly and a negative integer raises an error as expected. This is frustrating when programmatically calculated limit and expecting limit(0) to work as 'LIMIT 0' is valid sql.
Issue: Playhouse speedups
Author: stanep
State: closed
Created: 2016-03-02 21:46:16

Hi, using a new peewee (2.8.0) (pip install -U peewee) I got compiled playhouse._speedups file.
Now I am using SQL Server 2012 and pyodbc with peewee and so far it works but when I install latest version I got error running simple select statements.

File "\playhouse\speedups.pyx", line 179, in playhouse._speedups._QueryResultWrapper.iterate (playhouse\speedups.c:3690)
TypeError: expected tuple, got pyodbc.Row

If I rename _speedupds file I peewee works , so not sure if this is a bug, but maybe you can do
something so speedups will work with pyodbc :)


Just pass `use_speedups=False` when you are instantiating your Database object. That should address the issue. Peewee implements some cursor iteration helpers in Cython and expects the rows to be tuples, hence the issue you're seeing.

Let me know if the above fix addresses the problem.


Yes this fix the problem, thx
Issue: default_index cannot be None in IndexedFieldMixin
Author: IxDay
State: closed
Created: 2016-03-01 16:47:57

As in the constructor we setup the index like this: 
https://github.com/coleifer/peewee/blob/master/playhouse/postgres_ext.py#L129

``` python
self.index_type = index_type or self.default_index_type
```

I cannot specify at Field declaration that I do not want any index, as if I explicitly define `index_type` to be None it will be overridden by the previous line.

I think a better way to handle this can be to wrote the code like this:

``` python
class IndexedFieldMixin(object):

    def __init__(self, index_type='GiST', *args, **kwargs):
        kwargs.setdefault('index', True)  # By default, use an index.
        super(IndexedFieldMixin, self).__init__(*args, **kwargs)
        self.index_type = index_type

class ArrayField(IndexedFieldMixin, Field):

    def __init__(self, field_class=IntegerField, dimensions=1, index_type='GIN',
                 *args, **kwargs):
        self.__field = field_class(*args, **kwargs)
        self.dimensions = dimensions
        self.db_field = self.__field.get_db_field()
        super(ArrayField, self).__init__(*args, **kwargs)

...
```


This is fixed by 2fca64c482f536f280303a77fe8e9c040a0c96b3

To indicate that there should be no index, simply specify `index=False` when instantiating your `ArrayField`.


(the same goes for any other field that mixes in `IndexedFieldMixin`.)


Great thanks !


Thanks for reporting the issue, I appreciate it!
Issue: Create type
Author: IxDay
State: closed
Created: 2016-02-29 10:12:48

As I use custom types, is there a way to perform this in peewee or do I have to add a create_type method in the QueryCompiler class? If so, are you interested in the implementation of this feature ?

Here is an example of the code I use

``` sql
CREATE TYPE direction AS (title varchar(50), text text);
```


Unfortunately, you'll need to call `register_fields` to use custom types:

``` python
class DirectionField(...):
    db_type = 'direction'

PostgresqlDatabase.register_fields({
    'direction': 'direction',
})
```

Hope to get rid/clean-up that boiler plate in a later version of peewee but for now it is required.
Issue: peewee.OperationalError: database is locked using threadlocals and SqliteDatabase
Author: cryzed
State: closed
Created: 2016-02-28 21:35:09

Hey, I recently made this: [Feedbuffer](https://github.com/cryzed/Feedbuffer) and decided to use peewee for the database interface. I know it's not really your task to review my code, but I feel my problem would be easiest to explain if I linked you the file that manages the database communication [here](https://github.com/cryzed/Feedbuffer/blob/master/feedbuffer/database.py).

The thread architecture of the whole thing looks as follows: A CherryPy (webserver) instance is started in its own thread, which may spawn more threads (IIRC up to 10 more in the default configuration) to handle requests. Additionally I am using a ThreadPoolExecutor with 5 up to 40 workers, depending on the system the script is running on.

The problem occurs when I try to update many new feeds at once by using my RSS reader (imagine ~200 feeds with 10~ connections asking for updates). What puzzles me is that I am getting a "database is locked" error -- I was under the assumption that this could be entirely avoided by using `threadlocals=True` and instructing peewee with that to manage a single connection per thread. Am I doing something in my code that would require this mechanism to fail? Are those too many threads for sqlite to handle?

Some advice would be great.


I'm almost sure I fixed the problem [here](https://github.com/cryzed/Feedbuffer/commit/3b29434de92c8d0c47b90761858239ee1df08cd8). I'm just speculating, but I think the problem arose because I created the CherryPy server object in a [different thread](https://github.com/cryzed/Feedbuffer/commit/3b29434de92c8d0c47b90761858239ee1df08cd8#diff-5bc02cefb3ea9e27f1a6776eabd1935dL20) (main thread) instead of within the spawned thread. I also had to change the generator expression [here](https://github.com/cryzed/Feedbuffer/commit/3b29434de92c8d0c47b90761858239ee1df08cd8#diff-91c7a2706bca60a3f59aefad6b396b0eL32).

Now everything works without issues and it's reproducible -- reverting those changes leads to the aforementioned problems again and even slows everything down considerably. In the end I think it might be, because I am passing objects not local to the thread around, but I'd be lying if I said that I knew exactly what's going on. Maybe you have some input?

Thanks for peewee,
Chris


I'm not sure about your application or anything specifically you may be encountering, but what I can tell you is:
- sqlite enforces a _global database lock_ when you write to the db.
- multi-threaded apps need to be careful about serializing writes or correctly handling "busy" errors.
- using peewee with threadsafe=True is the default for a long time. all it means is you get a connection per thread, but those threads/connections can still contend for write locks.
Issue: get_or_create without autocommit
Author: Gagaro
State: closed
Created: 2016-02-28 13:12:29

Hello,

`get_or_create` and `create_or_get` fails if autocommit is set to False due to the use of a transaction inside these methods:

https://github.com/coleifer/peewee/blob/master/peewee.py#L4562
https://github.com/coleifer/peewee/blob/master/peewee.py#L4573

I created a test demonstrating this issue:

https://github.com/Gagaro/peewee/commit/cb64cce3e473187bc4284357941185088b3de57b

Also only these two methods create a transaction. Why is a transaction needed here?

Thanks


It creates an exception (or savepoint, if a transaction is present) to ensure the connection can be restored to a good state after an IntegrityError. Postgres, for instance, is rather unforgiving regarding this.

I'll add something to the docs, but for now I don't really see this as a bug.


In your example code, is there a reason you're not using the `atomic()` context manager / decorator? The `atomic()` helper will manage a stack of transactions/savepoints behind-the-scenes.


> In your example code, is there a reason you're not using the atomic() context manager / decorator?

I was reproducing my use case. It indeed works with atomic, but I can't use it for what I want to do.

Shouldn't atomic then also create a savepoint rather than create a new transaction when inside a begin ?


So I was able to have something working with the following:

``` python
self._transaction = db.transaction()
self._transaction.__enter__()
# [...]
self._transaction.__exit__(True, None, None)
```

It isn't very intuitive though.


Yeah, atomic will create a savepoint if there is a transaction in progress.

You can write your own `get_or_create()` that doesn't use a transaction/savepoint. Or you can use a decorator/context manager to manage your explicit transactions.
Issue: Query Error using APSW 
Author: mox1
State: closed
Created: 2016-02-26 21:20:42

When using the apsw_ext module, I receive the following error when performing queries:

```
>>> x = SomeModel.select().where(SomeModel.col1 == "test").exists()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "peewee.py", line 2896, in exists
    return bool(clone.scalar())
  File "peewee.py", line 2653, in scalar
    row = self._execute().fetchone()
AttributeError: 'apsw.Cursor' object has no attribute 'fetchone'
```

I see references in older versions of the code implementing a 'fetchone()' that calls next(), because APSW isn't DB-API compliant. Somewhere this code appears to have been re-factored out. 

This is using the trivial example model from the Docs

```
db = APSWDatabase(':memory:')

class BaseModel(Model):
    class Meta:
        database = db

class SomeModel(BaseModel):
    col1 = CharField()
    col2 = DateTimeField()
```

python 2.7.6 On Ubuntu 14.04
Using peewee 2.8.0 (git rev 4c1db1399c0906ea3ac59448fa4a333e813098fa)
Using python-apsw 3.8.2-r1-1ubuntu1 


0e055df5317e9966831b760a5a2aa8990f759207


Thank you for the excellent bug report, this should now be fixed.


You may need to update your apsw version, because in the latest versions it _does_ support `fetchone()`. I had to revert my commit that fixed the issue, due to MySQL python driver not implementing the iterator protocol on its cursor... Anyways, just try upgrading apsw and you should be set!
Issue: Document `playhouse` model paths
Author: MinchinWeb
State: closed
Created: 2016-02-25 21:05:30

I was working through the peewee documentation on [PickledField](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#PickledField). However, I could find nowhere in the documentation where to import this from.

I tried:
- the documentation page, near as I can tell, doesn't list this directly
- looking at the page documentation source, but there's no `current module` directive
- `PickledField` is listed in neither `dir(peewee)` or `dir(playhouse)`

This seems to be the case for all the classes listed under Playhouse.

I would recommend listing the import path somewhere in the documentation. Maybe just under each class:

> available at `playhouse.fields.PickledField`


Awesome, thanks!
Issue: Enable syntax highlight in Python snippet codes
Author: mazulo
State: closed
Created: 2016-02-25 04:19:18

Improve the read of the code


Thank you!
On Feb 24, 2016 10:19 PM, "Patrick Mazulo" notifications@github.com wrote:

> ## Improve the read of the code
> 
> You can view, comment on, or merge this pull request online at:
> 
>   https://github.com/coleifer/peewee/pull/869
> Commit Summary
> - Enable syntax highlight
> - Merge pull request #1 from mazulo/mazulo-patch-1
> 
> File Changes
> - _M_ README.rst
>   https://github.com/coleifer/peewee/pull/869/files#diff-0 (16)
> 
> Patch Links:
> - https://github.com/coleifer/peewee/pull/869.patch
> - https://github.com/coleifer/peewee/pull/869.diff
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/coleifer/peewee/pull/869.