akumoth/peewee-issues huggingface dataset:
Issue: How can I omit creation of a field but still have it in my model?
Author: maaaks
State: closed
Created: 2016-11-11 06:51:41

I'm using PostgreSQL. In one of my models I try to declare field named `tableoid`. This is a special field in PostgreSQL which is created automatically. So this always works:

```sql
select foo.tableoid from foo;
```

But the field cannot be created manually:

```sql
create table foo (tableoid int);
-- ERROR:  column name "tableoid" conflicts with a system column name
```

Yet I need to make Peewee aware of this field because when using [table inheritance](https://www.postgresql.org/docs/current/static/ddl-inherit.html) it is very convenient to join system tables and get tablenames by `tableoid`.

Could you please make something like `Field(..., omit_creation=True)` for this situation?

Maybe there is an easier way to do what I want, by somehow cheating in `MyModel.create()`, but I don't understand how.


Just added "undeclared" option, which was useful to me for a similar thing in SQLite regarding the "rowid" field. Commit:  eebbed7a5e1c6b3318bcbe6525ea80d900ee1f12
Issue: Postgres serial column but not primary key
Author: chuckhacker
State: closed
Created: 2016-11-10 22:02:59

`PrimaryKeyField` maps to the Postgres `serial` type, but it appears that there are indeed extra semantics forcing it to be a primary key (after all, one would guess that would be the case, given the name _PrimaryKey_Field).

Since I already have another field that is a primary key (but is not a serial, yeah, I know...), I tried overriding these extra semantics using `primary_key=False` (after all, to me fields seem like they should just be mappings to DB types with minimal semantics, but maybe that goes against the whole philosophy of an ORM). However, despite my best intentions, this didn't work:

```
chuck@excelsior ~/matchmaking_webrtc $ ./run.sh 
Traceback (most recent call last):
  File "main.py", line 122, in <module>
    class User(BaseModel):
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 4454, in __new__
    raise ValueError('primary key is overdetermined.')
ValueError: primary key is overdetermined.
```

How can I specify a field should be a `serial` without it being a primary key (I already have another primary key field)? Maybe we need to add a `SerialField` or the ability to set `primary_key` to `False` to override the extra implicit behavior of a `PrimaryKeyField` since it IS possible to do so in Postgres?

Adding a custom field is extremely easy: http://docs.peewee-orm.com/en/latest/peewee/models.html#creating-a-custom-field

I think that's probably what you'll want to do, add a `SerialField`.


Thanks, that's very helpful (although a trivial, common built-in type like `serial` should have already a `SerialField` in pg ootb IMO)!


Glad to have been helpful!
Issue: Make pskel script installable under usr/bin/
Author: stratakis
State: closed
Created: 2016-11-10 12:53:11

Currently since pskel script does not have the .py extension, nothing happens when invoking setup.py install with that script. It just exists at the sources. The simplest way to make this script get installed is by adding it at the scripts sections in setup.py, thus making it installable under usr/bin/.

Thanks!
Issue: aggregate_rows() [+ switch()] + get() don't play nicely
Author: alexlatchford
State: closed
Created: 2016-11-10 03:53:38

Not sure if this is a documented feature, but I'd love to use it if it was! Essentially prefetch for `get()` :D

If you're trying to aggregate multiple joins and backtracking using the `switch()` this results in some unexpected behaviour.

```python
from peewee import *

db = SqliteDatabase(":memory:")
db.connect()

class BaseModel(Model):
    class Meta:
        database = db

class MyModel(BaseModel):
    id = PrimaryKeyField()
  
class MyRelModel1(BaseModel):
    id = PrimaryKeyField()
    my_model = ForeignKeyField(MyModel, related_name="my_rel_models1")

class MyRelModel2(BaseModel):
    id = PrimaryKeyField()
    my_model = ForeignKeyField(MyModel, related_name="my_rel_models2")

MyModel.create_table()
MyRelModel1.create_table()
MyRelModel2.create_table()

m1 = MyModel.create()
MyRelModel1.create(my_model=m1)
MyRelModel1.create(my_model=m1)
MyRelModel1.create(my_model=m1)
MyRelModel2.create(my_model=m1)
MyRelModel2.create(my_model=m1)
MyRelModel2.create(my_model=m1)

my_model = MyModel.select(
    MyModel, MyRelModel1, MyRelModel2
).join(MyRelModel1).switch(MyModel).join(MyRelModel2).aggregate_rows().get()

print MyRelModel1.select().count()  # 3
print MyRelModel2.select().count()  # 3

print len(my_model.my_rel_models1)  # 1
print len(my_model.my_rel_models2)  # 1
```

Now if I try to access `my_model.my_rel_models1` it will only give back a single result instead of the 3 as expected.

I actually cannot get this to work either without the switch:

```python
my_model = MyModel.select(
    MyModel, MyRelModel1
).join(MyRelModel1).aggregate_rows().get()

print len(my_model.my_rel_models1)  # 1
```

Thanks,
Alex

PS. I know this is likely to cause performance issues when used on large datasets but the joins I'm looking to make are not huge 2-3 rows in each so figured I'd try this syntax to bundle up the network calls into a single query :)


This is documented, in ["Using aggregate_rows"](http://docs.peewee-orm.com/en/latest/peewee/querying.html#using-aggregate-rows) at the end of the section:

> Do not mix calls to `aggregate_rows()` with `LIMIT` or `OFFSET` clauses, or with `get()` (which applies a `LIMIT 1` SQL clause). Since the aggregate result set may contain more than one item due to rows being duplicated, limits can lead to incorrect behavior. Imagine you have three users, each of whom has 10 tweets. If you run a query with a `LIMIT 5`, then you will only receive the first user and their first 5 tweets.

Your best bet is to use `prefetch` with a `LIMIT 1` on the outer-most query.
Issue: the field can't named 'model'?
Author: msean
State: closed
Created: 2016-11-10 02:53:16

In Test Model,It set a field named 'model':
`class Test(peewee.Model):
    model = peewee.CharField(max_length=64,  null=True)

    class Meta:
        database = DATABASE
        db_table = 'test'
        schema = 'opr'
`
It happened error :
`File "/home/mwh/py3workplace/workplace/ad_tornado/handlers/opr.py", line 78, in post
yield from self.db.create(Test,model='android')
TypeError: create() got multiple values for argument 'model'`
I used peewee to related some Tables which had establishd and contains field 'model',but It happend error like this...if I don't change the field in db,is there any other ideas?
Issue: the field can't named 'model'?
Author: msean
State: closed
Created: 2016-11-10 02:53:07

In Test Model,It set a field named 'model':


class Test(peewee.Model):
        model = peewee.CharField(max_length=64,  null=True)
        class Meta:
            database = DATABASE
            db_table = "test"
            schema = "opr"


It happened error :
`File "/home/mwh/py3workplace/workplace/ad_tornado/handlers/opr.py", line 78, in post
yield from self.db.create(Test,model='android')
TypeError: create() got multiple values for argument ‘model’
`
I used peewee to related some Tables which had establishd and contains field 'model',but It happend error like this...if I don't change the field in db,is there any other ideas?


Please take the 10seconds and format your code...


In My Test Model,It set a field named 'model':

```
import peewee
class Test(peewee.Model):
    model = peewee.CharField(max_length=64,  null=True)
    class Meta:
        database = DATABASE
        db_table = 'test'
        schema = 'opr'
```

when I create a record,It happens an error:

```
File "/home/mwh/py3workplace/workplace/ad_tornado/handlers/opr.py", line 78, in post yield from 

    self.db.create(Test,model='android') TypeError: create() got multiple values for argument ‘model’
```

Weather the filed can named 'model'?@coleifer 
I used peewee to related some Tables which had establishd and contains field 'model',
if I don't change the field in db,is there any other ideas?


I think the issue is in your code. There is no "Database.create" method in Peewee. It appears to be a function in your own application. :astonished: :trollface:
Issue: Document (optional) requirements
Author: vkrizan
State: closed
Created: 2016-11-07 17:14:44

Please document (optional) requirements in one document (or file). This will help first-timers and help when packaging (RPM, deb).

Requirements for playhouse extensions (so far I've identified these):
```
Cython      # optional for speedups
simplejson  # optional for playhouse.kv and playhouse.sqlite_ext (json is supported since 2.6+ in standard lib)
apsw        # for playhouse.apsw_ext
bz2         # optional for playhouse.fields
zlib        # optional for playhouse.fields
Crypto      # optional for playhouse.fields
bcrypt      # optional for playhouse.fields
flask       # (obviously) for playhouse.flask_utils
psycopg2cffi  # optional for playhouse.postgres_ext
vtfunc      # optional for playhouse.sqlite_udf (old import at sqlite_ext ?)
gevent      # optional for playhouse.sqliteq
```

Database drivers (Already noted in [Vendor Specific Parameters](http://docs.peewee-orm.com/en/latest/peewee/database.html#vendor-specific-parameters) in docs), could be noted in a table of supported databases and required drivers.
```
Mysql: MySQLdb or pymysql
Postgres: psycopg2
Berkley: (optional) pysqlite2
```

Thank you. 

I can help to create such document(s), although I am not sure when or why `simplejson` is needed.

Certainly, thanks for the nudge.
Issue: Set PostgreSQL schema in atomic
Author: shanavar
State: closed
Created: 2016-11-05 20:46:29

Hey everybody,

so I got a problem regarding schema selection with peewee and PostgreSQL. I use peewee almost exclusively in transaction mode with `atomic`. While the schema is set to `none` in the meta of the class, I would want to set a specific schema in the `atomic` block. All queries in the atomic block should now act on the specified schema when the transaction of this `atomic` block happens. All the schemas in question do contain the same table structure. Is it possible to simply call `Model._meta.schema = myschema` at the beginning of the `atomic` block, e.g.:

```python
class BaseModel(Model):
    name = CharField()
    class Meta:
        schema = none
        db_table = users


with db.atomic() as txn:
    # set the schema for this block and the User Model
    User._meta.schema = myschema

    # this queries myschema.users and NOT public.users
    db.execute_sql('SELECT * FROM users WHERE name = %s', 'JohnDoe')

    # so does this
    User.create(name='JohnDoe')
```

So can I simply use `Model._meta.schema = myschema`? On the other hand, I also went through the documentation and a little bit through peewee source code and I think what I might need is some sort of `atomic` with properties of `Using` for schemas.

Yes, you can simply set `MyModel._meta.schema`. I'm not sure what it means to run a transaction against one schema...or how that works. I guess I thought a table lived in one schema, so the idea of switching it's schema to do a transaction should imply you're working with an entirely different table??


Yes. I am sorry, I did express myself the wrong way. What I wanted to say is, that I have one model that I use for multiple schemas so that all the tables do have the same structure but are of course different tables. And what I wanted is to set the schema within the atomic block, either for the whole transaction or per object/class. I wasn't sure if `User._meta.schema` wouldn't cause any problems if I use it in a peewee atomic block in a flask request or if it would simply set `tablename` to `schema.tablename` for queries inside this transaction only and have no effect otherwise.


You're fine to modify the schema attribute, peewee will use whatever's set when the query is executed.
Issue: Nested brackets in the documentation?
Author: NathanWailes
State: closed
Created: 2016-11-03 00:58:31

What do the nested brackets mean in the documentation?
`model_to_dict(model[, recurse=True[, backrefs=False[, only=None[, exclude=None[, extra_attrs=None[, fields_from_query=None]]]]]])`
([Link](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#model_to_dict))

I looked online and it seems that having brackets nested like that can be used to show that one optional argument requires the presence of another optional argument, but that doesn't seem to be the case here, as I've seen examples of calls to this function that specify (for example) the 'extra_attrs' parameter without some of the earlier ones.

These are optional keyword arguments. The arguments in brackets are shown with their default values (if they have one).


I understand that brackets indicate optional arguments; I'm asking why they're nested.


Because that's how you do it. https://docs.python.org/3/library/collections.html#collections.deque
Issue: Silence Signals Support
Author: smittysmee
State: closed
Created: 2016-11-02 15:41:20

Would it be possible to add the ability to silence signals for the pre_save handler, post_save handler, etc. This would allow for pre_save and post_save actions on the objects to prevent a loop modification within a handler.

Essentially, I have a process that once it is posted, the handler will take the object and perform a call within a module that takes the object, verifies it again our API, and modifies the object where need be.  Because it performs a save again when updating the object, I would like not run the call again to the API and save it again... etc. 

Proposed solution would be able to pass a kwarg to the save method that is overwritten by the playhouse signal method, that would look for the kwarg. If available and the value equates to true, then to skip over the signal method from running. 

I'm not following this at all. 


The signals all have a disconnect method, so you could use that. Alternatively you could use a global variable or thread local to keep track of the state.
Issue: FlaskDB with existing proxy
Author: timster
State: closed
Created: 2016-10-31 18:54:11

I have an existing set of models that I'm trying to integrate into a new Flask app. For all these models, I'm using a Proxy object as the database.

It would be great if FlaskDB accepted a Proxy instance as the `database` argument, but it seems like it only works with a Database instance.

Here's a minimal reproducible example:

```python
from flask import Flask
from peewee import Proxy
from playhouse.flask_utils import FlaskDB

app = Flask(__name__)

proxy = Proxy()
proxy.initialize('sqlite:///:memory:')

db = FlaskDB(app=app, database=proxy)

app.run()
```

Results in...

```
File "/opt/.virtualenvs/flask/lib/python3.5/site-packages/peewee.py", line 421, in __getattr__
    return getattr(self.obj, attr)
AttributeError: 'str' object has no attribute 'database'
```

Seems like the chance would need to be in the [_load_database](https://github.com/coleifer/peewee/blob/master/playhouse/flask_utils.py#L103) method. Or is there another/better way to use an existing proxy with FlaskDB?

Argh. This can be closed. After posting, I wondered why I was using FlaskDB in the first place. I updated my code to just connect/disconnect before/after app requests. No FlaskDB needed.

``` python
@app.before_request
def connect_proxy():
    initialize_proxy()
    proxy.connect()

@app.teardown_request
def close_proxy(exc):
    if not proxy.is_closed():
        proxy.close()
```
Issue: unsafe in multi-thread context
Author: argpass
State: closed
Created: 2016-10-31 03:30:50

```
class MyModel(Model):
    name = CharField()
    class Meta:
        db = my_db
```

using `MyModel` in multi-thread environment is unsafe, as db can not be shared in different threads.

Can you explain in greater detail? Peewee stores the connection state in a thread-local, so each thread should by default have its own connection.


Whatever, closing.


Please comment if you have any information you can provide to clarify the issue.
Issue: Add IntegrityError check in case post title is already in database.
Author: bdauer
State: closed
Created: 2016-10-27 20:02:54


Was just running through your awesome tutorial and caught an error when playing with it. Figured I'd submit a PR.


I appreciate the intentions, but your change-set has more problems than it attempts to solve. I've fixed the underlying issue with 6a07c8e1d949724ae10c9e304b1d7d30214cf2c6


Oh shoot I didn't copy over my whole change. I had added an additional line after the flash to render_template('create.html') in my local code, forked and made a quick change in github's editor. But reading over your fix I suspect there's a good reason why that's also not the best option. 


The reason it's not a good option (and whoops I need to fix something else, just remembered) is because you lose all your changes that you had been working on.


Oh actually nvm, I see that the template is pulling straight from request.form. Nevermind!
Issue: How do I make a CharField(collate='nocase')?
Author: stenci
State: closed
Created: 2016-10-25 21:35:35

Is it possible to create a field with [`COLLATE nocase`](https://www.sqlite.org/datatype3.html#collation_sequence_examples)?

```
CREATE TABLE tab (col TEXT COLLATE nocase);
```

I looked for something like this, but I couldn't find it:

```
class Tab(Model):
    col = CharField(collate='nocase')
```

The alternative is to make either a lowercase index or a collate nocase index, and make sure all the queries are either lowercase or collate nocase. It would be much easier to create the columns as collate nocase and get sorting and querying automatically case insensitive.


You can add a constraint:

``` python
column = CharField(constraints=[SQL('COLLATE NOCASE')])
```
Issue: MySQL - TimestampField Issue
Author: Byte-Lorenz
State: closed
Created: 2016-10-25 17:58:04

**Model:**
`class Issue(BaseModel):
time = TimestampField(resolution=1000)`

Doc: [http://docs.peewee-orm.com/en/latest/peewee/api.html#TimestampField](url)

**Issue:**
Inserting TimestampFied with value: `1475411420001` will be stored in the MySQL database as `1475411420001000`.
Although the resolution of the field is set to milliseconds (1.000) and not to microseconds (1.000.000)

When I'm reading out the data of the database and transforming it to the corresponding model with model_to_dict(...).  I'm encountering the following error: "ValueError: year is out of range". (on line 1283 of peewee.py)


> Inserting TimestampFied with value: 1475411420001 will be stored in the MySQL database as 1475411420001000.
> Although the resolution of the field is set to milliseconds (1.000) and not to microseconds (1.000.000)

It looks to me like you've over-thought it and done the multiplication yourself. The purpose of the `TimestampField` is to make all that unnecessary. `TimestampField` allows you to work with `datetime` objects in Python but store them as integers in the database. If you want to explicitly pass in a timestamp as an integer or floating point, then Peewee will scale that accordingly before storing it.
Issue: AttributeError with db.atomic() and SqliteQ
Author: theotherp
State: closed
Created: 2016-10-25 09:08:31

It's me again...

With 6ffc93cd867a7f3b28783ad4f533b1f9a205f540 I get an AttributeError using db.atomic() because lock_type is not set:

```
 File "C:/Users/user/PycharmProjects/nzbhydra\nzbhydra\search.py", line 318, in search
    with db.atomic():
  File "C:\Users\user\PycharmProjects\nzbhydra\libs\peewee.py", line 4471, in __enter__
    self.context_manager = self.db.transaction(self.transaction_type)
  File "C:\Users\user\PycharmProjects\nzbhydra\libs\playhouse\sqlite_ext.py", line 957, in transaction
    assert lock_type.lower() in ('deferred', 'immediate', 'exclusive')
AttributeError: 'NoneType' object has no attribute 'lower'
```

I'm using `with db.atomic("deferred")` to fix that but wanted to let you know.


Are you running on the current, very latest commit? Because `sqliteq`'s `atomic()` is kinda broken right now in a totally different way, as I'm in the process of implementing something like a pipeline in Redis.


Yeah, that's why I mentioned the commit hash. Not sure if you want feedback on your current development code.
Issue: Added a test to check whether a connection is already open.
Author: romanek-adam
State: closed
Created: 2016-10-24 09:51:44

Most importantly the check prevents from leaking database connections,
which can be disasterous when using connection pools.

Moreover it allows one to catch errors in the code which were caused
by not using database.connect() / database.close() properly.


I'd like to add something like this, but am worried about breaking too many people's code... I'll look into a similar patch.


9e76c999868197a4911fa2fa457c50ab42431ff1


Thanks!


FYI: this did break our code like you mentioned above. I can work around it in our codebase but I expect this caused problems for a lot of people; is there any more robust way of doing this?

Just run `database.get_conn()` which will only open a connection if the DB is closed.
Issue: "<<" operator does not function on Python sets
Author: josefdlange
State: closed
Created: 2016-10-21 22:06:57

The `<<` operator in a query where the right-hand side of a `.where()` clause is a Python `set()` causes for a `ProgrammingError` on query (at least via Postgres):

```
users = set([1, 2, 3])  # Assume the set is not as trivial as this...
for u in User.select().where(User.id << users):
    print u.name

ProgrammingError: can't adapt type 'set'
```

Of course, this works, but seems tedious:

```
users = set([1, 2, 3])  # Assume the set is not as trivial as this...
for u in User.select().where(User.id << list(users)):
    print u.name

Jimmy Neutron
Spongebob Squarepants
Rocky Balboa
Luke Skywalker
```

This issue in `psycopg2` suggests that higher-level abstractions should handle the adaptation from `set` to `list` in these cases.  Do you think `peewee` is adequately "high" or should applications using it be responsible?
https://github.com/psycopg/psycopg2/issues/340

I think the solution, if implemented in `peewee`, would go at/near the line linked below, casting `rhs` to a `list` if it is a `set`... but I wanted feedback before trying it and submitting a PR.
https://github.com/coleifer/peewee/blob/master/peewee.py#L1661


I think I'm going to pass on fixing this, as it's a bit of a slippery slope in terms of supporting every data-type. I'd suggest just doing as you showed, and coerce to list.


Can `Node._in` coerce to list automatically? [`list`](https://docs.python.org/2.6/library/functions.html#list) accepts any iterable, so the only thing the data-type needs to implement is the iterator.

That would work also with generators:

```
# Python 3
User.select().where(User.id << range(1, 10))
```