akumoth/peewee-issues huggingface dataset:
Issue: InsertQuery.execute quietly assumes synchronous cursor
Author: Elfoniok
State: closed
Created: 2018-01-12 14:37:19

When executing insert query, execute method assumes that synchronous code is executed.

If async cursor was used insted, method will silently return passed row, even though exception should be thrown.

Simple test, using SqliteQueueDatabase test this:

```
        model.Payment.create(primarykey="xyz", value=5)
        with self.assertRaises(IntegrityError):
           model.Payment.create(primarykey="xyz", value=5)
```

This should throw ```IntegrityError: UNIQUE constraint failed``` but silently passed payment is returned.

Have you tested with 3.0a? I just added a test-case to 3.0a which would indicate that this case is being handled properly:

```diff
diff --git a/tests/sqliteq.py b/tests/sqliteq.py
index ebda3b7..cd7aecb 100644
--- a/tests/sqliteq.py
+++ b/tests/sqliteq.py
@@ -69,6 +69,11 @@ class BaseTestQueueDatabase(object):
         self.assertRaises(OperationalError, curs.fetchone)
         self.database.stop()
 
+    def test_integrity_error(self):
+        self.database.start()
+        u = User.create(name='u')
+        self.assertRaises(IntegrityError, User.create, name='u')
+
     def test_query_execution(self):
         qr = User.select().execute()
         self.assertEqual(self.database.queue_size(), 0)
```

No this is from 2.10.2, when 3.0 will be released?

I tested this locally with Sqlite 2.10 and received the integrity error as expected:

```python
In [1]: from playhouse.sqliteq import *

In [2]: from peewee import *

In [3]: db = SqliteQueueDatabase('/tmp/testing.db')

In [4]: class Sample(Model):
   ...:     name = TextField(unique=True)
   ...:     class Meta:
   ...:         database = db
   ...:         

In [5]: db.start()
Out[5]: False

In [6]: Sample.create_table()

In [7]: u = Sample.create(name='s1')

In [8]: Sample.create(name='s1')
IntegrityError                            Traceback (most recent call last)
<ipython-input-8-1af0ec3b11a0> in <module>()
----> 1 Sample.create(name='s1')
....
     60             raise ResultTimeout('results not ready, timed out.')
     61         if self._exc is not None:
---> 62             raise self._exc
     63         self._ready = True
     64 

IntegrityError: UNIQUE constraint failed: sample.name
```

Yes i forgot to mention small thing, please add primary_key=True to Sample, and you see that it occur.
Issue: execution_context decorator throws error when used with Proxy (2.10.2)
Author: tadasant
State: closed
Created: 2018-01-09 19:27:58

Haven't dived too deep into this since the workaround is easy, but observed...

`models.py`
```python
from peewee import Model, Proxy

database = Proxy()

class MyObject(Model):
    id = CharField(primary_key=True)

    class Meta:
        database = database
```

`Wrapper.py`
```python
from src.models import database, MyObject


class Wrapper(object):
    @database.execution_context()
    def get_or_create_object(self):
        my_object = MyObject.get_or_create(id='1')
```

throws:

>   File "src/Wrapper.py", line 4, in DatabaseClientWrapper
    @database.execution_context()
  File "venv/lib/python3.6/site-packages/peewee.py", line 439, in __getattr__
    raise AttributeError('Cannot use uninitialized Proxy.')
AttributeError: Cannot use uninitialized Proxy.


Workaround is just:

`Wrapper.py`
```python
from src.models import database, MyObject


class Wrapper(object):
    def get_or_create_object(self):
        with database.execution_context():
            my_object = MyObject.get_or_create(id='1')
```

Maybe not worth fixing if it's just the conflicting nature of proxies vs. playing with decorators? Happy to take a shot at a PR if this is unintended
  

Python is interpreted at runtime. When Python evaluates the class definition it sees `@database.execution_context()`. A decorator implies a function call and since the `database` is not bound at the time Python constructs the class, you get the error. Not an issue, just Python doing its thing.
Issue: Cannot insert_many with optional fields
Author: lukasni
State: closed
Created: 2018-01-09 11:11:09

Model.insert_many() fails if the keys of optional fields are omitted in the dictionary, specifically if _some_ of the objects contain the optional keys and some do not:

Example
---------

```python
db = SqliteDatabase('test.db')

class MyModel(Model):
    my_id = PrimaryKeyField()
    required_field = CharField()
    optional_field = TextField(null=True)

    class Meta:
        database = db
```

```python
entity_list = [
    { 'my_id': 12345, 'required_field': 'Test', 'optional_field': 'Test Text'},
    { 'my_id': 12346, 'required_field': 'Test2'}
]

with db.atomic():
    MyModel.insert_many(entity_list).execute()
```

Observed Result
-----------------

`KeyError` triggered in `QueryCompiler.generate_insert()`, specifically [peewee.py#L2111](https://github.com/coleifer/peewee/blob/cacb86a2ee6c808ebda1062be53f5824aa847f48/peewee.py#L2111)

Expected Result
----------------

Records `12345` and `12346` created in table `test.mymodel`.

Might be achievable by simply changing the line linked above to `value = row_dict.get(field, None)`, preliminary test is promising.
  

Alternatively, row_dict would have to be generated for every row, which could result in a significant slowdown.

Actually, looking over it again, neither this nor the option above will work. The suggestion from the original issue will fail to insert optional fields if the first inserted row doesn't have all optional fields defined, generating row_dict for every row will result in uneven VALUES parameters, which will fail on insert.

This could be a decent substitute: 
```python
if not have_fields:
    fields = list(set().union(*(row.keys() for row in query._iter_rows())))
    fields.sort()
    have_fields = True
```

As you pointed out, a bulk insert query cannot have different values clauses as far as I know, so the difference has to be made up at query construction time... I'll give it some thought.

@coleifer I currently have it running with the above code on L2104 and onwards, passes tests and seems to perform as expected. Basically just builds a fields list using the set of all keys in the dict, only happens once per query, so the performance impact shouldn't be too huge, though it doesn't scale too well for very large insert sets.

> though it doesn't scale too well for very large insert sets.

Exactly the problem.

Check out dd0af77a51b660a8804c6c97115f57d9596e54f2 in 3.0a branch.

Think I'm going to close this one, as it should be resolved (generally) for 3.0a.
Issue: peewee .OperationalError: unknown tokenizer: icu zh_CN ( the same as #1371)
Author: Shazoo
State: closed
Created: 2018-01-04 11:15:59

Hi , 
I installed peewee via:
```bash
git clone ....; cd peewee
python setup.py build_ext
```

but it raise peewee.OperationalError: unknown tokenizer: icu zh_CN. 

I checked the issue 1371 https://github.com/coleifer/peewee/issues/1371

Tried the code mentioned , but it failed... 
```python
>>> import sqlite3
>>> conn = sqlite3.connect(':memory:')
>>> conn.execute('CREATE VIRTUAL TABLE thai_text USING fts4 (content, tokenize=icu th_TH);')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
sqlite3.OperationalError: unknown tokenizer: icu
```

so , anything wrong ? 

Thanks





You don't have the "icu" tokenizer, as the error states.

http://site.icu-project.org
Issue: MYSQL: TimeField returns datetime.time object instead of datetime.timedelta object
Author: mos87
State: closed
Created: 2017-12-31 12:22:42

I have the following construction:

```
class Config(Model):
    id = IntegerField(primary_key=True) # primary key = unique id
    date = DateField()
    time = TimeField()
    class Meta:
        database = db
        db_table = 'config'

def getRepetitiveIntervall():
    return Config.get(Config.id == 2).time
```

The time object is from type datetime.time instead of timedelta as it is in MYSQL
If i set the value in the DB to 100:00:00 (4 days, 4 hours) the peewee TimeField only returns only the 4 hours as a time object instead of a timedelta object with the 4 days...

Yes, this is documented. You can very easily add your own `IntervalField` implementation.

http://docs.peewee-orm.com/en/latest/peewee/models.html#creating-a-custom-field
Issue: Document exception hierarchy
Author: zmwangx
State: closed
Created: 2017-12-29 14:40:11

Sometimes I want to make sure I'm catching all possible peewee exceptions, but not knowing the base exception makes it hard. Having the exception hierarchy in API docs (inspired by [docs for built-in exceptions][1]) could be handy for this.

[1]: https://docs.python.org/3/library/exceptions.html#exception-hierarchy

Cheers!

Wow, that was fast. Thanks.

For what it's worth, the peewee exceptions are based on those already documented in the DB-API 2.0 spec. https://www.python.org/dev/peps/pep-0249/#exceptions

Good to know, thanks for the pointer.
Issue: Allow on=SQL() when joining a table
Author: maaaks
State: closed
Created: 2017-12-28 11:36:35

Currently, it's impossible to make a join with `on=SQL('true')`, only with `on=Clause(SQL('true'))`. I believe this doesn't make enough sense.

The problem is in `QueryCompiler.generate_joins()`.

You tried using Peewee 3.0a? I checked and it appears to work correctly.

No, not yet.
Issue: [Question] Custom setter\callback for model fields
Author: Nixellion
State: closed
Created: 2017-12-24 19:42:23

I'd like my application to send notifications whenever model is saved and certain fields are changed.

I tried doing it by using `@var.setter` but naturally I can not just create a setter for peewee field, so I'd have to make separate set of variable\parameters, which gets quite bulky.

I also read that I can override peewee Field classes with my own, but then I'll have to override each class, which does not seem like the best approach to me.

So what other options are there in peewee\playhouse for this?
Or which approach should I go with?

Again, I tried looking through the docs, and googled it, no luck so far.

You can use Model.dirty_fields, which tracks changes to fields. Will that work?

It could work, yes. However in a few functions I'm updating all fields and they all get dirty, even if information does not change.

So I ended up writing a function that uses `dict(self._data)` and compares current instance vs what's in the database before saving, and gives me only the fields where data is changed. So I'm doing it on save, and with dirty fields I'd have to move this check to another place.

Cool, I suppose that works, but if it's comparing against what's in the database then doesn't that introduce both an additional query to each save() *and* a race condition?

What's a race condition?

And yes, indeed it does, and I'm slightly concerned about it. However in this project I don't expect very heavy use of the database, it should work even with SQLite, and I'm sure that MySQL server will handle much more than this. But if there's another solutio to reliably check changes, and whether the field data is actually different from what it was before, that would be great.

Well, technically that's what `dirty_fields` is for. If you know which fields you're modifying and *don't* want them to be considered dirty, you can remove the field name from `model_instance._dirty` (a `set`). You can also do `model_instance._dirty.clear()` to reset the dirty fields at run-time. That's what I'd do. Anything more complicated gets pretty hairy.

Well, as I said, I basically have a web form, which allows user to modify every field of the row. Once commited it updates every field. I will still need to perform comparison with existing data somewhere.

I don't see what the harm is in saving most of the data in that case, but yea, I hear you.

There's no harm in saving it, it's the fact that I send notifications when something is updated, and I need to know which fields were changed exactly.
Issue: Document insert_many().execute() behavior
Author: vadimkantorov
State: closed
Created: 2017-12-22 19:35:50

I'd like to bulk insert new Users in a table (peewee 2.10.2).

I'm doing `User.insert_many([dict(display = 'u1'), dict(display = 'u2')]).execute()`. However, this returned boolean. How do I get primary id's for the newly inserted objects?

I saw in the docs that `insert(...).execute()` will return an id, so I expected the bulk version does the same.

Thanks!



It seems it's impossible to implement cleanly (`last_insert_rowid` etc) . So a note in the docs about the difference in behavior of `execute()` would be awesome.

What database engine are you using?

SQLite

Yeah sqlite doesn't support the returning clause so you're out of luck.
Issue: [Question] ForeignKey to self
Author: Nixellion
State: closed
Created: 2017-12-22 18:22:46

I'm trying to create a hierarchy of objects, and for that I need to have a foreign key to self class. Not to same instance, but to same class.

So basically what I need is this:

```
class Object(Model):
    parent = ForeignKeyField(Object, related_name='children, null=False)
```

But naturally it does not work, since Object does not yet exist at that moment.

How can I approach this problem? Could not find it in documentation, not sure how to call this issue even.

http://docs.peewee-orm.com/en/latest/peewee/models.html#self-referential-foreign-keys

Thanks a lot, sorry I could not find it myself.

No worries it's tricky if you aren't sure what to Google for.
Issue: Foreign key fields without related attribute
Author: vadimkantorov
State: closed
Created: 2017-12-22 00:06:02

I'd like to create a model that has two different foreign keys to another model. Peewee forces to use different `related_name`'s in order to make two different related Python attributes in the other model. In fact, I don't need those attributes.

Is there a way to tell Peewee to skip creating these attributes? I tried setting `related_name = None`, but it didn't help.

The usecase is modeling Audit records. I don't really care about conveniently retrieving all records by a given user using the Python code. I'll be just dumping all audit records to a spreadsheet and exploring it manually.

You can, in the model `Meta`, specify `validate_backrefs = False` to disable validation in 2.x.

If you're using 3.0a, validation is not enforced at the moment.

I had not considered supporting disabling backrefs altogether, but that might be a good idea to add in 3.0a.

In 3.0a, by specifying `backref='!'`, you can disable the addition of the backref accessor to the related model.
Issue: How to create temporary table in peewee??
Author: ptoyhawinf
State: closed
Created: 2017-12-20 09:17:26

HI
I used your API for a long time. And I want to create temporary table in my application, however I can’t find any describe in the document.
Could you give me some advice?

In peewee 3.0a you can pass `temporary=True` when calling create_table(). In 2.x you'll need to write the query by hand.
Issue: ImportError: cannot import name PooledPostgresqlExtDatabase
Author: toplinuxsir
State: closed
Created: 2017-12-20 07:48:00

Install peewee for python 2.7 on windows 

```
from playhouse.pool import PooledPostgresqlExtDatabase
```
raise exception : 
ImportError: cannot import name PooledPostgresqlExtDatabase

You probably need to install psycopg2 the postgres driver.
Issue: Support of BINARY fields
Author: conqp
State: closed
Created: 2017-12-14 09:11:18

I was just searching the internet for possibilities to handle BINARY fields (MySQL) in peewee and came across [this](https://github.com/coleifer/peewee/issues/801).
However, though the [respective pull request](https://github.com/coleifer/peewee/pull/802) was merged in 2015, I cannot find the respective code in peewee any longer and the official documentation also does not state any support of BINARY.
What happend to its support and can it be added again?

I believe you should be fine to use `BlobField` for most things, but it is also very easy to subclass `BlobField` and provide an implementation for the data-type you wish to use.

http://docs.peewee-orm.com/en/latest/peewee/models.html#creating-a-custom-field
Issue: Keep getting error: OperationalError: Database has not been initialized
Author: XuyangBai
State: closed
Created: 2017-12-11 11:04:40

Hi, I have following code using SqliteQueueDatabase. But I keep getting error `OperationalError: Database has not been initialized` I tried several methods but nothing works. So would please help me with that? Thanks you : )

dataService.py
```
from model import db
class dataService(object):
    def __init__(self, db_path):
        db.init(db_path)
        db.start()
    def some_func(self):
        pass
```
model.py
```
from playhouse.sqliteq import SqliteQueueDatabase
db = SqliteQueueDatabase(None)
class BaseModel(Model):
    class Meta(object):
        database = db

class Item(BaseModel):
    city = CharField()
    district = CharField()
```

Somehow I assume your dataService() object is not getting instantiated. There is a problem in your python code, not in Peewee.

Hello, I checked my code again and found that the error `OperationalError: Database has not been initialized` just thrown when I was trying to `from model import db`.  
So how could I import `db` without this error? ( Since my variable `db` and class `dataService` defined in two seperate files)
Issue: No way to create a table by string?
Author: josephmancuso
State: closed
Created: 2017-12-08 01:33:39

why is this not possible?

```
engine = database.engines['default']
migrator = MySQLMigrator(engine)

engine.create_tables(['users'], True) <--- not possible

migrate(
    migrator.add_column('users', 'column_name', 'CharField(default=255)'),
)

engine.close()
```

is there no way to simply create a table based off of a string? I want to separate the models and the migrations from each other so that I don't need to create a model before I create a table for it. I would like to create tables without a model

or even a way to do:

`migrator.create_table('users', True),`

What would it create if there were no columns defined?

it could just create a table with an id column by default

And what if you don't want the "id" column? Seems weird.

`migrator.create_table('users', pk='user_id'),` 


Peewee already has APIs for creating tables -- going to pass on these suggestions.
Issue: Model.get doesn't work when I pass a foreing key field attribute
Author: yanghg-basefx
State: closed
Created: 2017-12-04 07:43:25

Please see this code:

```python
import peewee
db = peewee.MySQLDatabase(**CONNECT_KWARGS)

class M(peewee.Model):
    parent= peewee.ForeignKeyField('self')
    class Meta:
        database= db
        db_table = 'M'

M.get(parent=None)  # It works.
M.get(parent=10000)  # It doesn't work. peewee.OperationalError: (1066, "Not unique table/alias: 't1'")
```

I've tested it both 2.10.2 and 3.0.a.

Good catch...the `filter()` API is not very smart when it comes to self-referential foreign-keys. I had thought it would handle this simple case, but apparently I'm missing a unit-test (which would've revealed the issue).

For now, try:

```python
M.get(M.parent == 10000)
```

I've pushed a fix for this simple case in 3.0a. Unfortunately, lookups that traverse the self-referential FK will still have the join alias issue.

It works fine! When the 3.0 branch become to stable, I will merge it into our company. Thanks!

But sorry, I can't really understand what does `lookups that traverse the self-referential FK will still have the join alias issue` mean. Can you give me a example? I can't reproduce it.

I've tried this:

```python
MA = M.alias()
MA.get(parent=10000)
```

It still work.

Or do you mean this?

```python
query = M.select().join(M)

M.execute()  # peewee.OperationalError: (1066, "Not unique table/alias: 't1'")
```

In fact, I always do the alias like your documentation [self-joins](http://peewee.readthedocs.io/en/latest/peewee/querying.html#self-joins)

```
MA = M.alias()
query = M.select().join(MA)

query.execute()
```

I think it is good enough, because if I want to specify other conditions, I can do that:

```python
M.select().join(MA).where(M.name == 'a', MA.name == 'b')
```

If you are doing a self-join you should always alias it (2nd example using `.join(MA)`).

The problem was that when using `.get()` with keyword-arguments, it is using the `.filter()` API under-the-hood, and this API does not cope well with self-referential FKs.

The standard APIs work just fine, however.

I got it, thanks!
Issue: ERROR:raysync:pysqlite or sqlite3 must be installed.
Author: nuxwork
State: closed
Created: 2017-12-04 01:58:39

The peewee i used , running ok on windows, but linux. it print `ERROR:pysqlite or sqlite3 must be installed.`

![20171204094931](https://user-images.githubusercontent.com/4026598/33532908-93616f10-d8d8-11e7-9339-a8a781e0f004.png)

os: CentOS-x86_64-minimal
python: Python 2.7.13
pip list:
altgraph (0.14)
backports-abc (0.5)
certifi (2017.7.27.1)
configparser (3.5.0)
dis3 (0.1.1)
future (0.16.0)
hashids (1.2.0)
macholib (1.8)
peewee (2.10.2)
pefile (2017.9.3)
pip (9.0.1)
protobuf (3.2.0)
PyInstaller (3.3)
pysqlite (2.8.3)
setuptools (20.3.1)
singledispatch (3.4.0.3)
six (1.11.0)
tornado (4.5.2)


Try opening up a python shell and running:

```
from peewee import sqlite3
print sqlite3
```

You can also try:

```
from pysqlite2 import dbapi2
import sqlite3
```

What happens?

My python is compiled by myself, I forget to add sqlite3 module.
Issue: execute_sql pass typle to MySQLdb if param is None
Author: daoiqi
State: closed
Created: 2017-12-02 12:38:53

https://github.com/coleifer/peewee/blob/c6b5946a997e39f9939b0261e2b75c46acbac8f2/peewee.py#L3831

pass typle to MySQLdb if param is `None`, why not `None` to MySQLdb?

?

MySQL

```python
sql = "select  DATE_FORMAT(MAX(NOW()),'%Y-%m-%d');   "
print database.execute_sql(sql)


sql = "insert into t_user set name='abc%abc';"
print database.execute_sql(sql)


```
```
Traceback (most recent call last):
  File "models.py", line 1332, in <module>
    print database.execute_sql(sql)
  File "/usr/lib/python2.6/site-packages/playhouse/shortcuts.py", line 203, in execute_sql
    sql, params, require_commit)
  File "/usr/lib/python2.6/site-packages/peewee.py", line 3454, in execute_sql
    cursor.execute(sql, params or ())
  File "/usr/lib64/python2.6/site-packages/MySQLdb/cursors.py", line 158, in execute
    query = query % db.literal(args)
TypeError: not enough arguments for format string

```

https://github.com/coleifer/peewee/blob/c6b5946a997e39f9939b0261e2b75c46acbac8f2/peewee.py#L3831

`params` is None

Why aren't you parameterizing your query? You should use %s as the placeholder for those params, then pass them in as a tuple:

```
db.execute_sql('select date_format(max(now)), %s)', ('%Y-%m-%d'))
```