akumoth/peewee-issues huggingface dataset:
Issue: How can I control retry times when I use sql_error_handler to retry SQL execution?
Author: messense
State: closed
Created: 2015-08-27 03:27:39

I am using the RetryDatabase example code I found in peewee test cases.

https://github.com/coleifer/peewee/blob/db328731034e45fb4f8a1a2a4a562da1dc0d1f23/playhouse/tests/test_database.py#L183

The problem is that I want to control retry times manually. How can I do that in a nice way?


Figure out a way myself. Sample codes:

``` python
import logging

from playhouse.pool import PooledMySQLDatabase
from peewee import OperationalError


logger = logging.getLogger(__name__)


class RetryPooledMySQLDatabase(PooledMySQLDatabase):

    def sql_error_handler(self, exception, sql, params, require_commit):
        if isinstance(exception, OperationalError):
            self.close()
            self.connect()
        return True

    def execute_sql(self, sql, params=None, require_commit=True):
        retry_times = 0
        max_retry_times = 3
        while True:
            try:
                return super(RetryPooledMySQLDatabase, self).execute_sql(
                    sql, params, require_commit
                )
            except OperationalError:
                if retry_times >= max_retry_times:
                    logger.warning(
                        'Maximum retry times(%s) exceeded, give up retrying.',
                        max_retry_times,
                    )
                    raise

                retry_times += 1
                logger.warning(
                    'Executing SQL(%s), params(%s) error, retrying %s/%s time',
                    sql,
                    params,
                    retry_times,
                    max_retry_times,
                )
```
Issue: ascii codec error while running execute_sql
Author: roozbehid
State: closed
Created: 2015-08-21 22:00:35

I have a simple execute_sql command like this 

```
    self.db.execute_sql("""
    INSERT IGNORE INTO artists (name,alias_name)
                               select name,alias_name
    from my_actors""")
```

The first time I run it (which I mean there are no duplicates...) it runs fine.
But second times it give me 

```
       File "c:\WinPython\python-2.7.6.amd64\lib\site-packages\peewee.py", line 3191, in execute_sql
cursor.execute(sql, params or ())
       File "c:\WinPython\python-2.7.6.amd64\lib\site-packages\MySQLdb\cursors.py", line 207, in execute
if not self._defer_warnings: self._warning_check()
       File "c:\WinPython\python-2.7.6.amd64\lib\site-packages\MySQLdb\cursors.py", line 117, in _warning_check
warn(w[-1], self.Warning, 3)
       File "c:\WinPython\python-2.7.6.amd64\lib\warnings.py", line 29, in _show_warning
file.write(formatwarning(message, category, filename, lineno, line))
       File "c:\WinPython\python-2.7.6.amd64\lib\warnings.py", line 38, in formatwarning
s =  "%s:%s: %s: %s\n" % (filename, lineno, category.__name__, message)
      UnicodeEncodeError: 'ascii' codec can't encode characters in position 17-20: ordinal not in range(128)
```

Running this command on MySQL works fine and I think this has to do something with peewee trying to check for warning and .....
I am using peewee 2.6.3 and MySQL charachter sets for all tables are set to utf-8.


Using an interactive debugger, are you able to examine the message of the warning that is failing to be displayed? My feeling is that this is not an issue with Peewee, but I'd like to be sure before closing.


``` python
file.write(formatwarning(message, category, filename, lineno, line)
```

That is the line that I think is problematic. Something in the warning message, maybe?


warning is about having duplicate values I guess, that is why I used INSERT IGNORE. but anyway this throws an exception which should not happen.
The name of column I am using is unicode and somethimes it is ascii I guess...I am myself having problem printing it out....so I used two different lines of code to check if it is unicode or it is ascii!

But all subsequent enquires and everything works as far as I can tell! warning displaying causes problem


> The name of column I am using is unicode and somethimes it is ascii I guess...I am myself having problem printing it out....so I used two different lines of code to check if it is unicode or it is ascii!

OK, then that sounds like the issue. You might've mentioned that in the issue description. This isn't a peewee issue, but an issue with the MySQLdb library.
Issue: Make it possible to use ``upsert`` and ``on_conflict`` on MySQL
Author: messense
State: closed
Created: 2015-08-14 05:56:05

Ref #684

I am not sure how to write a corresponding test case for now.

And I don't know if it is ok to use `isintance(meta.database, MySQLDatabase` here. What do you think?


Cool, this is a nice start.


This behavior was added in #753.
Issue: Examples of how to add request hooks in wheezy.web
Author: thirashima
State: closed
Created: 2015-08-11 16:45:23

I'd like to see an example of how to add request hooks for db connection management to the wheezy.web framework. I've tried to figure it out myself, but it is non-obvious to me. Thanks!


Don't know the framework, but you should probably use [middleware](https://pythonhosted.org/wheezy.http/userguide.html#middleware). Assuming `peewee_middleware` is never the last one:

```
def peewee_middleware(request, following):
    db.connect()
    try:
        response = following(request)
    finally:
        if not db.is_closed():
            db.close()
return response

app = WSGIApplication(middleware=[
    lambda x: peewee_middleware,
    path_routing_middleware_factory
])
```

I would think this question belongs more to wheezy.web project than to peewee, though.


Thanks. I added this here because the peewee documentation specifically asks you to open a ticket if there are any other frameworks you'd like to see examples for. http://docs.peewee-orm.com/en/latest/peewee/database.html#other-frameworks


Oh, I stand corrected. Did you have luck with the suggestion?


@thirashima -- If you come up with some working code, please comment on this issue and I will add it to the docs. For now I am closing this.


the code that tuukkamustonen provided does seem to be working. thank you everyone for the help.


Cool, I will update the docs, thanks d00dz.


http://docs.peewee-orm.com/en/latest/peewee/database.html#wheezy-web
Issue: Does peewee support MySQL REPLACE INTO syntax?
Author: messense
State: closed
Created: 2015-08-08 04:23:26

I only see upsert support for SQLite in documentation.

And another question, does peewee support MySQL `INSERT IGNORE` or `INSERT...ON DUPLICATE KEY UPDATE` syntax?


Not at present, but I would consider a pull request and would work with you on getting the code into good shape. For now I'd suggest using hand-written SQL queries if you want upserts with MySQL.
Issue: Support for the MySQL BIT data type
Author: jmenglund
State: closed
Created: 2015-08-05 08:45:31

http://dev.mysql.com/doc/refman/5.7/en/bit-type.html

http://stackoverflow.com/questions/289727/which-mysql-datatype-to-use-for-storing-boolean-values

For my own databases I would probably not use the BIT data type at all (see for example http://www.xaprb.com/blog/2006/04/11/bit-values-in-mysql/) but I need it in order to work with a database not created by me.


At present Peewee uses the `bool` data-type, which is also what Django uses for BooleanField in Django.

If you want `Bit` specifically you can either create a custom field, or specify a field override:
- http://docs.peewee-orm.com/en/latest/peewee/models.html#creating-a-custom-field
- http://docs.peewee-orm.com/en/latest/peewee/api.html#Database


Thank you very much! My problem was easily solved.

I'm sorry I didn't read the docs more carefully. Perhaps a link to the section "Creating a custom field" directly after the field types table could prevent others from making the same mistake.


Added the links, thanks!


@jmenglund Hello, I am trying to do the same but I have troubles converting the string of hex characters retrieved to booleans. I am a beginner in Python, could you share some snippet
Issue: Unexpected behavior doing nested iteration on a SelectQuery
Author: martinburch
State: closed
Created: 2015-08-04 17:36:50

Hi, I wanted to bring an issue to your attention: nested iteration of SelectQuery results doesn't work as expected. You can see an example, and a workaround, on my StackOverflow question:
http://stackoverflow.com/questions/31815235/python-peewee-iterate-over-selectquery

Is there an iterator that needs to be reset somewhere? Maybe at the end of the inner loop?

Thanks!


Here is a description of the problem:

``` python
# create 5 users, named user0-user4
query = User.select()
for o in query:
    print 'o ->', o.username
    for i in query:
        print '    i ->', i.username
```

This will print:

```
o -> u0
    i -> u0
    i -> u1
    i -> u2
    i -> u3
    i -> u4
```


So when you iterate over a `SelectQuery` it creates a `QueryResultWrapper` which allows for efficient iteration and indexing. The problem is that the iterator is stateful.

The workaround I'd suggest is just to coerce to a `list` beforehand. You can also call `QueryResultWrapper.fill_cache()`.

I'll see about a better fix.


This should now be fixed in master.


Great, thanks!
Issue: Add multi-column unique constraints support for Model.create_or_get
Author: messense
State: closed
Created: 2015-08-03 08:17:28


Could probably tidy up the code a little, and we'll need sign off from @coleifer on the `query` conditional, but assuming those two things are signed off, then LGTM.


@messense Has this already been merged into master?

I don't know. I haven't used peewee for a long time and I think I won't be working on this so I closed it.

@coleifer Any thoughts on whether you'd like this in master?

No, get or create is evil. One of my biggest regrets
Issue: reflection module is using a not convenient import statement
Author: aliubchenko
State: closed
Created: 2015-07-31 16:26:05

Reflection.py has this as the import statement:
from peewee import *

When you include peewee in your own package of libraries and use it, all classes from peewee become <your_package>.peewee.*
However, for models that were created using Introspection, peewee classes are still peewee.\* because of that import statement.

Consequently if you want to do a select query on the generated models, in this function:

```
def parse_node(self, node, alias_map=None, conv=None):
    sql, params, unknown = self._parse(node, alias_map, conv)
    if unknown and conv and params:
        params = [conv.db_value(i) for i in params]

    if isinstance(node, Node):
```

The last check fails. Because it is comparing node (peewee.Entity, created by Introspection) with <your_package>.peewee.Node
This fails and leads to queries missing the "as t1" statement right after "from <table>" clause.
Leading to the database complaining about missing column names t1.<column>.

The fix that worked for me is changing the import statement to use relative import:
from ..peewee import *

This allowed everything to reside in my package and the check worked and proper query was generated.


I don't really think this is a use-case I had intended to cover.
Issue: [Question] Why can't I assign a new model instance to a model's foreign-key field
Author: EPadronU
State: closed
Created: 2015-07-30 15:50:28

## Sample layout

``` python
class File(database.Model):
  pass


class User(database.Model):
    file = ForeignKeyField(File)
    pass
```
## The code that reproduces the _odd_ behavior

``` python
>> user = User.get(id=1)
>> user.file = File()
>> user.file.save()
>> user.save()
>> user = User.get(id=1)
>> user.file
None
```
## The behavior with an existing instance

``` python
>> file = File.get(id=1)
>> user = User.get(id=1)
>> user.file = file
>> user.save()
>> user = User.get(id=1)
>> user.file.id
1
```
## Observations
- The behavior in the first situation should be similar to that of the second.
- Could the lack of an ID in the first situation be the cause of the _odd_ behavior?
## Pip freeze

```
blinker==1.3
Flask==0.10.1
Flask-Login==0.2.11
Flask-Mail==0.9.1
Flask-Markdown==0.3
Flask-Principal==0.4.0
Flask-Security==1.7.4
Flask-WTF==0.12
ipython==3.2.1
itsdangerous==0.24
Jinja2==2.7.3
Markdown==2.6.2
MarkupSafe==0.23
passlib==1.6.2
peewee==2.6.3
Werkzeug==0.10.4
wheel==0.24.0
wtf-peewee==0.2.3
WTForms==2.0.2
```

---
## Update - Thu Jul 30 15:55:24 UTC 2015

I just put under test my first observation and it looks like I was right (maybe):

``` python
>> file = File()
>> file.save()
1
>> user = User.get(id=1)
>> user.file = file
>> user.save()
>> user = User.get(id=1)
>> assert user.file is not None
```


That's an interesting point and has to do with the implementation of the descriptor. When the value is set (you assign the `File()` obj), the file has no primary key so peewee records `None` as the related object ID. The fact that the `File` later gets an ID is not caught by the field.

I think this warrants some investigation, thanks for bringing it up.
Issue: pylint complains no-value-for-parameter on Object instatiation...
Author: danthedeckie
State: closed
Created: 2015-07-30 15:19:34

Python 2.7, creating objects e.g.:

``` python
    mike = Person()
```

results in pylint complaining with `no-value-for-parameter`, missing `names`, `bases`, `attrs`.


wontfix
Issue: ArrayField does not accept tuples
Author: bdoms
State: closed
Created: 2015-07-29 07:02:18

So I love peewee. It's great. No problems so far, but I did just run into this strange behavior. Using an ArrayField for postgres from the playhouse (which is an awesome extension) and I had a function that output several values and I tried to use those directly. Something like:

``` python
def foo():
    return a, b, c

BarModel.create(array_field=foo())
```

In Python I generally expect that all iterables are treated more or less the same, but clearly that's not happening here because I then got this error:

``` bash
peewee.ProgrammingError: column "array_field" is of type integer[] but expression is of type record
LINE 1: INSERT INTO "barmodel" ("array_field") VALUES ((1, 2, 3)) RE...

HINT:  You will need to rewrite or cast the expression.
```

I understand the workaround: just cast it. But it's inconvenient and error prone for me to have to remember to do that every time. I realize that this may be more of a feature than a bug, but having to write `list()` around every function that returns what's already essentially a list seems super un-Pythonic to me. I suspect that this error appears for more non-list iterable types than just tuple as well, but I didn't test anything else.

Anyway, it would be really nice if we could use tuples and custom iterators to generate values for an array field. Here's a full test case (assuming an equivalent database and user set up, obviously):

``` python
from peewee import PostgresqlDatabase, Model, IntegerField
from playhouse.postgres_ext import ArrayField

DB = PostgresqlDatabase('test', user='test_user', password='test_password', port=5433)


class BarModel(Model):
    class Meta:
        database = DB

    array_field = ArrayField(IntegerField, default='{}', null=True)


def run_test():
    if BarModel.table_exists():
        BarModel.drop_table()
    BarModel.create_table()

    # this works just fine
    array_field = [1, 2, 3]
    BarModel.create(array_field=array_field)

    # this causes an error
    array_field = (1, 2, 3)
    BarModel.create(array_field=array_field)


if __name__ == '__main__':
    run_test()
```
Issue: Prefetch syntax
Author: alexlatchford
State: closed
Created: 2015-07-29 03:25:15

Hey Charles,

I've been writing a few methods with prefetch and find it incredibly useful but I wondered why the syntax is different to the regular, non-prefetch solution and whether it's possible to merge the two approaches.

For example:

``` python
class User(Model):
    name = CharField()

class Message(Model):
    user = ForeignKeyField(User)
    data = TextField()

# Create models and populate data ...
```

Existing (contrived to simplify) prefetch selection solution.

``` python
users = User.select()
messages = Message.select()

data = []
for user in prefetch(users, messages):
    data.extend(format_messages(user))
else:
    print data

def format_messages(user):
    return [message.data for message in user.messages_prefetch]
```

This works great but my dilemma is that now I want to do this:

``` python
user = User.get(User.id == user_id)
data = format_messages(user)
```

And this results in an `AttributeError` on `messages_prefetch` :(

The question I have is how come the prefetch and non-prefetch syntax differ? Surely it's lazy evaluation so either the data is there or it needs to be fulfilled. Is this just an oversight or am I missing something more fundamental? I'm guessing the latter.

What I'm envisaging is that prefetch just uses the regular syntax.

``` python
def format_messages(user):
    return [message.data for message in user.messages]
```

Sorry for the annoying question, but it's been bugging me a little while. We use this pattern a fair amount for RESTful GET calls in both single/multiple and I'd like to be able to reuse the same formatting functions.


This is to differentiate pre-fetched "static" results from the descriptor that exposes related objects as a `SelectQuery`.

I do not plan on changing this, but you can modify your code:

``` python
def format_messages(user):
    messages = getattr(user, 'messages_prefetch', user.messages)
    return [message.data for message in messages]
```
Issue: [Bug?] Relationships involving a one-to-one relationship throw an exception
Author: EPadronU
State: closed
Created: 2015-07-28 23:19:38

## Sample layout

```
class User(database.Model):
    pass


class Patient(database.Model):
    user = ForeignKeyField(User, primary_key=True)


class Disease(database.Model):
    pass


class PatientDisease(database.Model):
    patient = ForeignKeyField(Patient)
    disease = ForeignKeyField(Disease)
```
## The code that throws the exception

```
>> patient = Patient.get(user=1)
>> patient.patientdisease_set
```
## Output | head -n 6

```
$PWD/.venv/lib/python2.7/site-packages/peewee.pyc in _parse(self, node, alias_map, conv)
   1546             if conv and isinstance(conv, ForeignKeyField):
   1547                 params = [
-> 1548                     conv.to_field.db_value(getattr(node, conv.to_field.name))]
   1549             else:
   1550                 params = [node._get_pk_value()]

AttributeError: 'User' object has no attribute 'user'
```
## Observations
- This also happens with one-to-many relationships
- This not happens in the other way. For example: `patient_disease.patient`
## Pip freeze

```
blinker==1.3
Flask==0.10.1
Flask-Login==0.2.11
Flask-Mail==0.9.1
Flask-Markdown==0.3
Flask-Principal==0.4.0
Flask-Security==1.7.4
Flask-WTF==0.12
ipython==3.2.1
itsdangerous==0.24
Jinja2==2.7.3
Markdown==2.6.2
MarkupSafe==0.23
passlib==1.6.2
peewee==2.6.3
Werkzeug==0.10.4
wheel==0.24.0
wtf-peewee==0.2.3
WTForms==2.0.2
```


Thanks for reporting, fixed in c1030f58c0df67b28fefa75182d6a93698921e01


Thanks for fixing it so quickly. Cheers.
Issue: PasswordField
Author: ronyb29
State: closed
Created: 2015-07-28 18:33:25

I think i got something going on,
but please see comments on files


This looks good, I'm sorry for not responding sooner, completely fell off my radar.

I'd like to merge. Looks like the only issue is the lack of the unicode literal in the earlier python 3 build. If that can be fixed, I'd be happy to merge this.


Nice! :)
@coleifer any rough idea when this will be released? 


Hopefully the next week or so. Thank you!
On Nov 12, 2015 9:46 AM, "Yohan Boniface" notifications@github.com wrote:

> Nice! :)
> @coleifer https://github.com/coleifer any rough idea when this will get
> released?
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/coleifer/peewee/pull/672#issuecomment-156142700.
Issue: Unable to make custom Field as inner class of an Enum
Author: fluffy-critter
State: closed
Created: 2015-07-26 10:35:14

This might be a peculiar use case, but for better encapsulation I tried doing this:

```
from peewee import *
from enum import Enum
class Foo(Enum):
    foobar = 0
    qazquux = 1
    bleck = 2
    class Field(IntegerField):
        def db_value(self,value):
            return value.value
        def python_value(self,value):
            return Foo(value)

class TableWithFoo(BaseModel):
    stored_foo = Foo.Field()
```

which generates a `TypeError: 'Foo' object is not callable` error. However, if I add a `@staticmethod` decorator to the declaration of `class Field`, it works just fine (and behaves correctly).

The `@staticmethod` decorator is a suitable workaround but it seems inelegant. In other testing I've found that `Enum` base classes allow inner classes without that hack, so I'm guessing the problem lies with subclassing `IntegerField` although I haven't tried creating a custom class with a custom `Field` class inside of it to verify that.

Granted this is all a silly thing to just avoid having FooField as a top-level class, which makes no difference to the functionality or the readability of the code, but it just feels like it should be possible. :)


This has nothing to do with Peewee as far as I can tell, but is an implementation detail of the `Enum` class.


Okay, thanks.

Do you have any suggestions for a more Pythonic solution than the (somewhat confusing) `@staticmethod` decorator, or am I just overthinking this? I'm still learning Python and want to establish good habits. :)


Use @staticmethod.


Thanks!

## 

Sent from my tin cans and a piece of string. 

> On Jul 27, 2015, at 9:34 AM, Charles Leifer notifications@github.com wrote:
> 
> Use @staticmethod.
> 
> ―
> Reply to this email directly or view it on GitHub.
Issue: Add support for citext columns in postgres 
Author: esafak
State: closed
Created: 2015-07-22 21:51:11

`pwiz` incorrectly translates `citext` columns to `UnknownField`.


I wouldn't say "incorrectly" -- Peewee does not have any field comparable to a case-insensitive text field. Pwiz is designed to cover the most common data-types but it cannot hope to cover them all. Hence, `UnknownField`. You can go through and replace them with `TextField` if you like.


Aren't `text` and `citext` different, though? Isn't the real problem that peewee doesn't have a case-insensitive field?


They are different, and yes, the issue is that peewee does not have a case-insensitive text field. That said, I don't plan on adding one. If you'd like you can add one yourself quite easily, though:

http://docs.peewee-orm.com/en/latest/peewee/models.html#creating-a-custom-field


Is it as simple as,

    class CITextField(TextField):
        field_type = 'citext'

That's all you need.