akumoth/peewee-issues huggingface dataset:
Issue: MySQL Error 1059(FK identifier name is too long) on migrator.add_column
Author: Dushyant7
State: closed
Created: 2018-11-29 10:37:53

peewee==2.8.1

I am using MySQLMigrator to migrate my tables.

When I am creating new column using add_column which is ForeignKeyField.
It throws `peewee.InternalError (1059, Identifier name ... is too long)`.

The reason behind this is MySQL has limit of 64 characters for FK identifier and identifier name which is created by add_column is more than 64 characters (68 in my case using format "fk_%s_%s_refs_%s").
https://dev.mysql.com/doc/refman/5.7/en/identifiers.html

I am not sure if I can specify such identifier in ForeignKeyField or in add_column function.

Not sure about other, but specific to MySQL, FK identifier name is auto generated in case not specified. This can solve the issue.

There should be some tests for maximum length constraint on identifiers.

Peewee contains code that correctly truncates long index names. I've added a new patch that truncates long foreign-key constraint names, which will be included in the next release.

You're using an ancient version, I suggest upgrading.
Issue: Can not do cast chaining
Author: lephuongbg
State: closed
Created: 2018-11-29 02:25:40

Reproducible snippet:

```python
import peewee as p

node = p.ColumnBase()
node.cast('TEXT').cast('JSONB') # Will throw TypeError: 'str' object is not callable
```

I will make a PR soon for this.

Thanks, nice catch. Merged your patch.
Issue: get raw sql without parameters
Author: dridk
State: closed
Created: 2018-11-24 21:32:18

I would like to create a view from a select statement.
Actually, I m trying this : 

    rawsql = Variant.select().where(Variant.pos > 3000).sql()[0]
    db.execute_sql(f"CREATE VIEW test AS {rawsql}" ) 

Unfortunally, with SQLITE it returns 

    OperationalError: parameters are not allowed in views


How can I get the raw sql without parameters ?

      WHERE value = 3 instead of  WHERE value = ?  , 3 

At present Peewee's schema manager does not have support for views.

Just write out the SQL. In the time you took to type up your issue, and for me to respond, surely you could have typed out the SQL?

If you insist, you can implement a custom query component.

Here's a complete example:

```python

from peewee import *
from peewee import ColumnBase

class AsIsValue(ColumnBase):
    def __init__(self, value):
        self._value = value
        super(AsIsValue, self).__init__()

    def __sql__(self, ctx):
        if isinstance(self._value, str):
            # Ensure strings are quoted.
            value = "'%s'" % self._value
        elif self._value in (True, False):
            # Convert boolean to integers.
            value = 1 if self._value else 0
        elif self._value is None:
            value = 'NULL'
        return ctx.literal(str(value))

db = SqliteDatabase(':memory:')

class User(Model):
    username = TextField()
    is_admin = BooleanField(default=False)
    class Meta:
        database = db

db.create_tables([User])

for username, admin in [('huey', True), ('mickey', False), ('zaizee', True)]:
    User.create(username=username, is_admin=admin)

admin = (User
         .select()
         .where(User.is_admin == AsIsValue(True))
         .order_by(User.username))

view_sql = 'CREATE VIEW "user_admin" AS %s' % admin.sql()[0]
db.execute_sql(view_sql)

Admin = Table('user_admin', ('id', 'username', 'is_admin'))
for obj in Admin.select().execute(db):
    print(obj['username'])
```

Now...which is easier? Up to you.

I did something like this and it works like a charm. I am Not sure it is elegant 


```python
class Variant(Model):
	class Meta:
		database = db 

	@classmethod
	def create_view(cls,name, where_clause):
		sql = Variant.select().where(where_clause).sql()

		# replace parameters in raw sql  
		raw    = sql[0]
		params = sql[1] 

		for param in params:
                        # quote params 
			param = f"'{param}'"
			raw = raw.replace("?", param, 1) 

		Variant._meta.database.obj.execute_sql(f"CREATE VIEW {name} AS {raw}")
		# Create a class dynamically
		class VariantView(cls):
			class Meta:
				db_table = name

		return VariantView

Variant.select().count() // return 590

MyView = Variant.create_view("my_view", Variant.pos > 432234 ) 
MyView.select().count() // return 121 


```

The use of formatting strings is just vomit, in my opinion...yuck. Whatever works man. I'm glad I'm not maintaining that mess, though.
Issue: Heuristics for determining whether to use python_value is incorrect if uses with CTE having same column with model
Author: lephuongbg
State: closed
Created: 2018-11-21 10:32:15

Reproducible snippet:

```python

class Products(Model):
    id = IntegerField()

tmp = Products.select(Products.id.cast('text')).cte('tmp', columns=('id',))

# Normally, we will #join with tmp here but use #from_ instead just for demonstration purpose
Products.select(tmp.c.id).with_cte(tmp).from_(tmp)
# => [{'id': 1}] <-- should be string here instead
```

The problem is due to the change of this line:

https://github.com/coleifer/peewee/blob/178bc0155bf569c9245296e90ba5066901e208bc/peewee.py#L6483

```diff
- raw_node._coerce
+ node._coerce
```


I believe the relevant changes are here: ecfa98c9f2d5d6b30f238d97928662d909f769e2

Although I would point out that changing the above to "raw_node" does not affect the outcome of the query in any case. Change the line and try it for yourself -- the value is still an int.

There is no connection, internally, between `tmp.c.id` and either `Products.id` or `Products.id.cast('text')`. Dynamic column references such as `tmp.c.id` bear no relation to any field, and the heuristic simply looks at the column names to try and determine if a dynamic column lines up with a field.

The way to handle this correctly is:

```python
Products.select(tmp.c.id.coerce(False)).with_cte(tmp).from_(tmp)
```

And in fact, without the change you referred to (`node._coerce` vs `raw_node._coerce`) this logic would not work.

So the code is correct, in my opinion. See this test for details: 0d2e81b873c6ac6fe968779c3bf18af06f5a6a47

Thanks for clarifying. Though I should say that it might be a breaking change, because it breaks my code after upgrade to newer Peewee. May be a note in CHANGELOG?

> because it breaks my code after upgrade to newer Peewee

Are you sure?

Yes. It is one of our company's project and upgrading peewee to master (we also had another problem that was also fixed in master), our CI just broke all over the place that I had a hard time to debug it. Probably because we make use of CTE a lot.

But indeed, changing our code to what you suggested works so it's ok for us now. 

Gotcha gotcha -- I think I was confused because when testing locally, changing `node` to `raw_node` did not have any effect in the example you provided. However, there are some other reorganizations of that code that may be causing the discrepancies you're seeing.

At any rate, using the `.coerce(False)` ought to take care of it. Thanks for your comments and contributions -- it's nice to work w/someone who has a solid understanding of how peewee works.
Issue: In an expression, don't use left hand side's converter if right hand side's is a function
Author: lephuongbg
State: closed
Created: 2018-11-21 02:34:45

Should fix #1791 

I chose to implement this a bit differently -- the result is equivalent for your test-case, however. Rather than checking the rhs inside expression, I thought it made sense to simply not apply a conversion to the arguments of a function at all.

178bc0155bf569c9245296e90ba5066901e208bc
Issue: Peewee incorrectly uses json.dumps on parameters in edge-case
Author: lephuongbg
State: closed
Created: 2018-11-21 01:49:45

Not sure how to make the title better but here is a small reproducible snippet:

```python
import peewee as p
from playhouse.postgres_ext import *

db = PostgresqlExtDatabase(
    # The usual config
)

class TestModel(p.Model):
    class Meta:
        database = db
    metadata = BinaryJSONField()

# OK
TestModel.select(p.fn.jsonb_build_object('key', 'value')).execute()
# => SELECT jsonb_build_object('key', 'value') FROM "testmodel" AS "t1"

# NOT OK, see the 'key' is now '"key"', and 'value' is now '"value"' instead
TestModel.select(TestModel.metadata.concat(p.fn.jsonb_build_object('test', 'value'))).execute()
 # =>  SELECT ("t1"."metadata" || jsonb_build_object('"test"', '"value"')) FROM "testmodel" AS "t1"
```

Thanks for this, and for the patch. I chose to implement the fix a bit differently than your suggested fix, but they should be equivalent. The solution I merged is more general, and avoids applying conversion to function arguments in any case.

178bc0155bf569c9245296e90ba5066901e208bc

Thanks. I also did think that this should be put somewhere else better. 

P/S. How could you keep being responsive to every issues all the time like this? (No need to answer, just me admiring you).
Issue: How's graphene-peewee going?
Author: yanghg-basefx
State: closed
Created: 2018-11-20 03:48:09

I see [this ticket](https://github.com/graphql-python/graphene/issues/289), and I want to use peewee with graphql, but I didn't find any other info. How's it going? Or I must switch to sqlalchemy to use graphene-sqlalchemy?

I'm not pursuing it, nor am I in a position to recommend an alternative.

I ported @insolite’s [graphene-peewee-async](https://github.com/insolite/graphene-peewee-async) to the regular Peewee. Check it out here:

[graphene-peewee](https://github.com/emperorplatform/graphene-peewee)

This is my first major OSS contribution, so please leave feedback if anything can be improved. Also, this is an early release with limited testing. It has not been used in production by me. There may well be bugs.
Issue: logger.debug
Author: adontz
State: closed
Created: 2018-11-19 02:35:51

I am pretty sure logger.debug is called wrong way all across the code
I have found lines like this

    logger.debug((sql, params))

in peewee.py and playhouse/apsw_ext.py, but it really should be

    logger.debug(sql, params)

because method signature is

    debug(msg, *args, **kwargs)

and one should pass format string and arguments for that format string as separate arguments, not a tuple. You can find an usage example in docs

    FORMAT = '%(asctime)-15s %(clientip)s %(user)-8s %(message)s'
    logging.basicConfig(format=FORMAT)
    d = {'clientip': '192.168.0.1', 'user': 'fbloggs'}
    logger = logging.getLogger('tcpserver')
    logger.warning('Protocol problem: %s', 'connection reset', extra=d)

Also test_utils.count_queries contains

    q.msg[0].startswith('SELECT ')

while it really should be

    q.msg.startswith('SELECT ')

Again, you can find in docs that "msg – The event description message, possibly a format string with placeholders for variable data." and not tuple.

The logger statements are intended to log the tuple. Interpolation/parameterization wouldn't work, anyways, as not all databases necessarily use "%s" to signify parameters...sqlite uses "?" for instance. The idea is that we log both the SQL and the params as a tuple rather than trying to interpolate.

I understand that
('SELECT c FROM t WHERE d > ?', [3])
is more preferable than
('SELECT c FROM t WHERE d > 3')
and more correct. I am not saying you can or should interpolate, probably you cannot, or that arguments should not be logged, they should be.

What I am saying is that nobody ever expects LogRecord.msg to be anything but string. All log files are screwed.

> What I am saying is that nobody ever expects LogRecord.msg to be anything but string. All log files are screwed.

You have to explicitly enable logging for peewee, and even then, at the DEBUG level. If you are trying to log peewee at the debug level, presumably you know what you're doing.
Issue: [Request] Raise exception on missing parens for field type definitions
Author: ghost
State: closed
Created: 2018-11-18 21:03:15

When one forgets to add parentheses to a field type definition. e.g.
 mtype  = IntegerField 
instead of 
 mtype  = IntegerField ()
and create_tables executes, tables from the models get built, but columns do not get created for fields w/o parentheses. Exceptions aren't raised either.

It would be nice, if peewee alerted the negligent programmer to his/her mistake(s).

Going to pass.
Issue: OperationalError when creating table with schema in SQLite
Author: belidzs
State: closed
Created: 2018-11-17 20:00:33

I have the following model:

```python
db = Proxy()


class BaseModel(Model):
    class Meta:
        database = db
        schema = 'api'


class Koltseg(BaseModel):
    id = CharField(primary_key=True, max_length=8)
    kotesszam = CharField(max_length=10)
    fokonyv = CharField(max_length=50)
    kozbeszerzeses = BooleanField()
    afas = BooleanField()
    osszeg = DecimalField(max_digits=12, decimal_places=2)
```

Normally I use postgres, but during testing I instantiate an in-memory SQLite database and use that. However, when I call `create_tables` from the test fixture I receive an exception complaining there is no database with the name 'api'.

```python
@pytest.fixture
def app():
    db = SqliteDatabase(':memory:')
    db.bind(MODELS, bind_refs=False, bind_backrefs=False)
    db.connect()
    db.create_tables(MODELS)
    app = create_app(testing=True, test_db=db)
    app.config['TESTING'] = True

    yield app  

    db.drop_tables(MODELS)
    db.close()
```

And the output:

```
test setup failed
self = <peewee.SqliteDatabase object at 0x0000015CA7DDA9E8>
sql = 'CREATE TABLE IF NOT EXISTS "api"."koltseg" ("id" VARCHAR(8) NOT NULL PRIMARY KEY, "kotesszam" VARCHAR(10) NOT NULL, "...v" VARCHAR(50) NOT NULL, "kozbeszerzeses" INTEGER NOT NULL, "afas" INTEGER NOT NULL, "osszeg" DECIMAL(12, 2) NOT NULL)'
params = [], commit = True

    def execute_sql(self, sql, params=None, commit=SENTINEL):
        logger.debug((sql, params))
        if commit is SENTINEL:
            if self.in_transaction():
                commit = False
            elif self.commit_select:
                commit = True
            else:
                commit = not sql[:6].lower().startswith('select')
    
        with __exception_wrapper__:
            cursor = self.cursor(commit)
            try:
>               cursor.execute(sql, params or ())
E               sqlite3.OperationalError: unknown database "api"
```

I'm using peewee 3.7.1

Schemas, in SQLite, refer to attached databases. Therefore, to use "schemas" in SQLite you will need to attach databases with the given names. This is an advanced usage and probably not what you want, so I recommend avoiding the use of schema with sqlite unless you know what you're doing.

Thanks for the quick answer!

I'm really only using SQLite to simulate my production environment (postgres) as close as possible during testing. In my production database tables are scattered through several schema (however my flask app only uses one). 

If you say it's not possible to just replace the database driver and use it like that while testing I can live with that but I think it would be nice if the documentation would reflect this limitation.
Issue: Fixed deprecation warning for importing Callable ABC's from collections
Author: AVOstap
State: closed
Created: 2018-11-17 13:00:51

to avoid:

```bash
.../peewee.py:169: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Callable
```

Merged equivalent patch.
Issue: model_to_dict fails to process disabled backrefs 
Author: adontz
State: closed
Created: 2018-11-16 18:06:16

If there is a model with disabled backref "ForeignKeyField(OtherModel, backref='+')" then model_to_dict fails to convert such model to dict, because model has no '+' attribute

    File ".../site-packages/playhouse/shortcuts.py", line 105, in model_to_dict
        descriptor = getattr(model_class, foreign_key.backref)
    AttributeError: type object 'MyModel' has no attribute '+'

https://github.com/coleifer/peewee/blob/cb6cc1d6bc224070b6deef81dc314fb44c7e8386/playhouse/shortcuts.py#L105
As far as I understand foreign_key.backref should be tested like this
https://github.com/coleifer/peewee/blob/cb6cc1d6bc224070b6deef81dc314fb44c7e8386/peewee.py#L4569

LINK: somehow related to https://github.com/coleifer/peewee/issues/1405

Nice, thanks.

You response speed in incredible! Thank you!

Can you please clarify why you have "not in '!+'" in one place and "== '+'" in another place?
Are not backref='+' and backref='!' the same?

Sorry for the lack of documentation on this... "!" is retained for backwards-compatibility but should not be used. At some point I will remove that and just use `== '+'`.
Issue: model_to_dict backrefs=True, but exclude_autogenerated, i.e. *_set
Author: patarapolw
State: closed
Created: 2018-11-15 13:41:54

Is this possible? Otherwise, I would have to `backrefs=False` altogether, and manually `extra_attrs`.

There is no "exclude_autogenerated" option. You can, however, use the "exclude" option to specify the foreign-key of the backref you wish to exclude.
Issue: Documentation on advanced connection management
Author: DaanVanVugt
State: closed
Created: 2018-11-15 12:43:12

Hi,

Thanks for this cool package. I've been trying to work with multiple processes, and find references here and there to http://docs.peewee-orm.com/en/latest/peewee/database.html#advanced-connection-management but can't seem to find this page anymore. Is this not supported anymore?

Thanks,
Daan

That information is outdated and was likely removed when 3.0 was released... I don't see any references to that from within Peewee.

For multiprocess, just be sure not to connect until after the fork.
Issue: fix InterfaceError when postgresql connection is closed
Author: velom
State: closed
Created: 2018-11-07 22:53:24

when `self._state.closed is False` and `self._state.conn.closed != 0`

I'm unclear on why this patch is necessary. Granted, Peewee could do a lot more introspection to detect whether the connection is closed or not, but this method is intended more to refer to the "peewee" connection state rather than the "driver" connection state.

I'm going to pass on this -- connections dying unexpectedly probably *should* raise an `InterfaceError`.

I have described my situation at https://github.com/coleifer/peewee/pull/1779#issuecomment-436895926.
During some exceptions, `self._state.conn.closed` became non zero, but `self._state.closed` is still `False`.

> I'm going to pass on this -- connections dying unexpectedly probably _should_ raise an `InterfaceError`.

By the way, when connection is dying, peewee doesn't switch `self._state.closed` to `True`.
So every next request will fail here
```python
def cursor(self, commit=None):
        if self.is_closed():
            self.connect()
        return self._state.conn.cursor()
```
because `self.is_closed()` returns `False`, but connection is dead (`self._state.conn.closed != 0`)

Correct - it is necessary to implement reconnect logic.

do you think, there is no bug in peewee in this case?
Issue: peewee order_by failed
Author: hanvanos
State: closed
Created: 2018-11-07 13:36:27

I installed a new version of Ubuntu and with that also peewee version3.7.1
Now the order_by meta declaration in the models does not work anymore.
I had to add order_by statements in the queries.

The update from 2.x to 3.x has a lot more changes than just that:

http://docs.peewee-orm.com/en/latest/peewee/changes.html
Issue: PostgreSQL UPDATE statement with FROM clause with subquery support
Author: nilp0inter
State: closed
Created: 2018-11-04 12:34:39

This is a second try on #1775 , because the request was accepted and then reverted I am opening this.

The changes with master are:

1. I reverted the SET part in the UPDATE test queries.
2. I wrote a helper function to recursively qualify only ColumnBase member of Expressions.
3. I applied this helper to the RHS of the assignments in the SET part of UPDATE queries.

All tests, including the new regressions tests appear to be passing now.

Let me know what you think.

I merged a similar patch that is roughly equivalent: 2018e579ac81db5113a26e55ce9078cc80bab446

I have also removed all references to QualifiedNames in the tests -- the helper is no longer needed as the query-builder handles it correctly now. 

Awesome, thanks!
Issue: text field type when using mysql/mariadb database
Author: hegusung
State: closed
Created: 2018-11-01 01:05:29

Hi,

According to the documentation, TextField should use MySQL longtext column type but it uses the text column type. Is this the desired behavior and the documentation has not been changed? If so, how can I change my code to make sure it uses a longtext and not a text field type?




Apologies -- the docs are incorrect. Peewee used "longtext" up until 3.x, at which point it changed to just plain old "text".

To have a longtext field:

```python
class LongTextField(TextField):
    field_type = 'LONGTEXT'
```

Then just use it as you would a regular `TextField`.
Issue: Add update_or_create() helper method
Author: willyhakim
State: closed
Created: 2018-10-30 15:55:36

get_or_create is amazing! Could we add some more super powers by implementing update_or_create()

No. `get_or_create` is a hack and I don't advise anyone to use it under any circumstances. It's vulnerable to race-conditions and weird behavior. Just write the extra 2-3 lines of code and do it right. Wrap it up in a classmethod on the model it pertains to. You can thank me later.
Issue: ArrayField of ForeignKeyFields, is it possible?
Author: droserasprout
State: closed
Created: 2018-10-30 15:44:19

Hi!
I'm working with postgresql and peewee 3.7.1 and trying to initialize model's field as an array of ForeignKeyFields this way:

```python
PreviousTasks = ArrayField(ForeignKeyField, {'model': 'self', 'column_name': 'previous_tasks', 'field': 'ID', 'backref': 'TaskPreviousTasks'})
```
Docs say ArrayField argument field_class gets "a subclass of Field" so this should be possible theoretically. But I'm getting this error:
```python
  File "/opt/app/include/models/tickets.py", line 176, in <module>
    class Task(ModelBase):
  File "/opt/app/include/models/tickets.py", line 191, in Task
    PreviousTasks = ArrayField(ForeignKeyField, {'model': 'self', 'column_name': 'previous_tasks', 'field': 'ID', 'backref': 'TaskPreviousTasks'})
  File "/usr/local/lib/python3.6/site-packages/playhouse/postgres_ext.py", line 155, in __init__
    self.field_type = self.__field.field_type
  File "/usr/local/lib/python3.6/site-packages/peewee.py", line 4552, in __getattr__
    if attr in self.rel_model._meta.fields:
AttributeError: 'str' object has no attribute '_meta'
```
Is there any way to do this kind of trickery or should I use PreviousTasks as an array of integers and fetch referenced models manually?

I wouldn't call that trickery, I'd call it ... well, nevermind.

Just use an actual junction table. Play to the strengths of your relational database.

Sorry to resurrect an old thread, but if anyone (like me) is looking for a similar thing: have a look at the [ManyToManyField](http://docs.peewee-orm.com/en/latest/peewee/api.html#ManyToManyField)