akumoth/peewee-issues huggingface dataset:
Issue: Field class doesn't have an `index_type` attribute
Author: ananis25
State: closed
Created: 2019-02-11 14:20:52

The ModelIndex class looks for the index_type argument in the definition of a Field when creating an index for it, but it is not defined as an attribute for the Field class.

```
class ModelIndex(Index):
    def __init__(self, model, fields, unique=False, safe=True, where=None,
                 using=None, name=None):
        self._model = model
        if name is None:
            name = self._generate_name_from_fields(model, fields)
        if using is None:
            for field in fields:
                if isinstance(field, Field) and hasattr(field, 'index_type'):
                    using = field.index_type
```

Might it be an oversight? If available, creating indices of type other than the default (B-tree) for fields would be really simple. 

```
class Table(Model):
    columnA = DateTimeField(index_type = 'BRIN')
```

Also, thank you for the great library :)

This is only used internally by some special fields in the `playhouse.postgres_ext` module. However you *can* specify a custom index type via the `using=` parameter of `ModelIndex`:

```python
class Table(Model):
    timestamp = DateTimeField()

Table.add_index(Table.index('timestamp', using='BRIN'))
```

I *do* think this is a legit suggestion that it should be applicable to all field types... I'm going to look at expanding the scope of the `index_type` bit to maybe apply across all fields.

Fixed -- now it is possible to write:

```python
column_a = DateTimeField(index=True, index_type='BRIN')
```

Thank you! The commit diff shows it was clearly more involved that it looks. 

The `add_index` method was also doable but for my use-case, I needed to redefine the Model class every time I used it. Another hack that worked was:

```
class Table(Model):
    columnA = DateTimeField(index = True)
    columnA.index_type = 'BRIN'

    class Meta:...

```
Issue: Add support for column name option in ForeignKeyField
Author: michaellzc
State: closed
Created: 2019-02-10 05:22:08

```py
class User(Model):
	user_id = IntegerField(primary_key=true)

class Tweet(Model):
	tweet_id = IntegerField(primary_key=true)
	reply_to = ForeignKeyField('self') # Reference to itself
```
Current behaviour of the above snippet will automatically change column name of `reply_to` to `reply_to_id`. AFAIK `ForeignKeyField` class does not expose `column_name` parameter like `Field` class does.

It will be great if we can parametrize `column_name` for all `Field` class.

Here is what I have in mind.

```py
class Tweet(Model):
	tweet_id = IntegerField(primary_key=true)
	reply_to = ForeignKeyField('self', column_name='reply_to')
```

ForeignKeyField *does* support `column_name`. What are you talking about?
Issue: [docs]fix optimistic locking document in hacks.rst
Author: ymym3412
State: closed
Created: 2019-02-09 09:37:57

In peewee3, Model class has no attribute `_dict` , but has `__dict__` .
And in hack.rst, there is no definition of `ConflictDetectedException` . 

Reffered to this blog post. 
http://charlesleifer.com/blog/optimistic-locking-in-peewee-orm/

Thx.
Issue: Map result of raw query to a model
Author: IvaYan
State: closed
Created: 2019-02-08 17:12:29

Hello!
Is it possible to map result of a raw query to a particular Model, assuming that I know the structure of the values returned by the query? Something like this:

```py
class MyModel(Model):
       # declare fields here, as usual
       value1 = peewee.IntegerField()
       value2 = peewee.IntegerField()

result = database.execute('SELECT value1, value2 FROM ...').map_to(MyModel)
for m in result:
    print(m.value1, m.value2)
```

The `result` variable contains model instances instead of raw tuples.

In that case you can do:

```python
result = MyModel.raw('SELECT value1, value2 FROM ...')
for m in result:
    print(m.value1, m.value2)
```
Issue: Support for different blob types in mysql
Author: maxnoe
State: closed
Created: 2019-02-08 08:43:08

I would need to store some binary data larger then 65kb in a mysql database.

peewee only seems to support one `BlobField`, which maps to mysqls `blob` which is max 65kb.

How can I make sure, the table is created with a `MEDBLOB` or `LONGBLOB` datatype while also maintaining sqlite compatiblity for local testing?

Just subclass and specify the data-storage class:

```python
class LongBlogField(BlobField):
    field_type = 'LONGBLOB'
```

This is not compatible with using a proxy that can either be a sqlite or mysql, as sqlite has no longblob type, or am I mistaken?

https://www.sqlite.org/datatype3.html

From 3.1.3:

> If the declared type for a column contains the string "BLOB" or if no type is specified then the column has affinity BLOB.
Issue: readonly field support
Author: SteveByerly
State: closed
Created: 2019-02-08 05:04:57

I created a `BaseTrackingModel` to capture some tracking data on insert/update.

```python
class BaseTrackingModel(BaseModel):
    created_at = DateTimeField(null=False, constraints=[SQL('DEFAULT CURRENT_TIMESTAMP')])
    updated_at = DateTimeField(null=True, constraints=[SQL('ON UPDATE CURRENT_TIMESTAMP')])
```

I quickly ran into an issue since I was not setting the `created_at` field, which I naively expected to be set by the database.

Some operations have an `only` param to define which fields to persist. The `insert_many` method uses a fields mapping which can accomplish the same behavior. 

However, the `bulk_create` method has no similar option - though I was successful in overriding the `bulk_create` class method to filter out these field names.

I feel like this is a core feature I would want built-in to the base Field, where I wouldn't need to worry about every point where data is persisted.

Do you have any thoughts or previous details on this type of functionality? Would you be able to point me in the right direction of where you could imagine the functionality living?

Not exactly answering the issue, but rather than use `bulk_create()` why not just use `insert_many()`?

I tend to prefer creating the instances of my objects directly. It's easier for me to see which object I'm creating and see that I'm assigning values properly. I also find this method a bit easier to search and refactor.

`insert_many` gets close with dictionaries, but you're still one-step removed since the fields (dict keys) have no direct relation to the object within the code. the line where the dict is created might be several lines apart from the `insert_many` call using the model class.

I dunno...specifying default values using database constraints is totally fine, but it doesn't play very well with an active-record ORM like Peewee. Unless you are using Postgres, which supports INSERT...RETURNING, you would need to issue a subsequent SELECT to obtain the values that are being stored in these columns. You might consider using the `insert_many()` APIs.

So I've been kicking this around for a few days. The main APIs that would be impacted by such a change would be `save()`, `create()` and `bulk_create()`. My concern is that implementing something that causes certain columns to be excluded from these calls will also require a separate mechanism to override this exclusion. Since there already exist APIs for performing arbitrary insert/update queries, I think I'll pass on implementing this kind of functionality.
Issue: Get timestamp from database as int
Author: jansedlon
State: closed
Created: 2019-02-06 12:25:54

I have a column of type Timestamp (or DateTime). If I save it to database, it's saved as integer, but when I want to get it from the database, it's returned as datetime. Is there any way to get it as it is in the database (integer) ?

So you should:

* Use `DateTimeField` if you want to work with `datetime` in python and store the data using the `datetime` data-type in your sql db.
* Use `TimestampField` if you want to work with `datetime` in python and store the data using an integer in your sql db.
* Use `IntegerField` if you want to work with timestamps as integers in python and store the data as an integer in your db.
Issue: Cannot use Proxy connection_context decorator.
Author: maxnoe
State: closed
Created: 2019-02-04 13:24:19

Defining methods or functions with the `connection_context` decorator does not work with Proxies.

It throws `AttributeError: Cannot use uninitialized Proxy`.

Although I see no reason, why this should be necessary at definition time.

```python
from peewee import Proxy, Model, TextField

database = Proxy()

class User(Model):
    username = TextField(unique=True)

    class Meta:
        database = database

@database.connection_context()
def get_all_users():
   return list(User.select())

```

While `Proxy` is primarily indicated for use with deferred database objects, it is in a sense a general abstraction. Adding such a method to `Proxy` produces a bit more tight-coupling to `Database`, which I'd prefer to avoid.

The issue is that `database.connection_context()` is evaluated as an import-time side-effect. And at import-time we don't necessarily have an initialized Proxy, as you point out.

We actually could rewrite the code as:

```python
database = Proxy()

def get_all_users():
    return list(User.select())

get_all_users = database.connection_context()(get_all_users)
```

When we do that, it's clear that we're calling a function on the uninitialized proxy, asking that it be evaluated immediately. I know you understand all this, but I figure it's worth writing out as I'm thinking this through.

So a way to work around this would be to use the context-manager instead, within the function:

```python
def get_all_users():
    with database.connection_context():
        return list(User.select())
```

Also note that `connection_context()` isn't the only database decorator you might use. There are also:

* `database` (the object itself supports use as a decorator or context manager)
* `database.atomic` and related functions

I think my preference will be to create a subclass, `DatabaseProxy`, which will implement placeholders for connection_context as the other decorators.
Issue: WHERE TRUE condition
Author: againagainst
State: closed
Created: 2019-02-03 17:28:20

Is there any possibility to make the simple `TRUE` condition for where clause? I would like to use it as an initial condition and combine with some extra conditions if necessary:
```
    def get_users(self, removed=False, deactivated=False):
        cond = Model.TRUE
        if removed:
            cond &= ~(Users.removed >> None)
        ...
        return [to_dict(user) for user in Users.select().where(cond)]
```

Eh, you can do `Value(1) == Value(1)`...but the way I would handle that is to accumulate a list and then use `reduce`, e.g.:

```python
from functools import reduce
import operator

# ...

expr_list = []
if removed:
    expr_list.append(User.removed.is_null(False))
if deactived:
    expr_list.append(User.active.is_null(True))
# ...
query = User.select()
if expr_list:
    query = query.where(reduce(operator.and_, expr_list))
return query
```

A couple notes on your code, by the way:

* `~(Users.removed >> None)` is NOT the same as `IS NOT NULL`. It is `NOT (... IS NULL)`. Better to use `User.removed.is_null(False)` which translates to Users.removed IS NOT NULL.
* It is recommended never to use plural for model names. Better to use `User`.
* Returning a query is always more flexible. Rather than return a list of dicts, return a query, and only convert to a list-of-dicts at the last possible moment.
Issue: Add support for subselects with CTEs.
Author: iksteen
State: closed
Created: 2019-01-30 17:04:23

By delaying the emission of CTEs and emitting them in the subquery's context, they are contained within its surrouding parentheses. To avoid adding extra parentheses to the CTE queries, force the context for the CTE list to not have the subquery flag.

This fixes #1809.

I've merged an equivalent patch, which expands on your own and adds comments and test-cases: 6707199769e0f8878988242bca347f577e3f0e7b

Thank you very much for digging into this and finding such a concise solution.
Issue: Common Table Expression and prefetch
Author: iksteen
State: closed
Created: 2019-01-29 17:04:17

Consider the following example (note that this is obviously a silly example, but just enough to show the problem):
```
from peewee import *
db = PostgresqlDatabase('scoreboard')

class Base(Model):
    class Meta:
        database = db

class Challenge(Base):
    pass

class Attachment(Base):
    challenge = ForeignKeyField(Challenge, backref='attachments')

cte = (
    Challenge
    .select(fn.MAX(Challenge.id).alias('max_id'))
    .cte('demo')
)

q = (
    Challenge
    .select(Challenge)
    .join(cte, on=(Challenge.id == cte.c.min_id))
    .with_cte(cte)
)
p = Attachment.select()
r = prefetch(q, p)
print(list(r))
```

The generated SQL for the prefetch of Attachment will be:
```
SELECT "t1"."id", "t1"."challenge_id" FROM "attachment" AS "t1"
WHERE (
    "t1"."challenge_id" IN
    WITH "demo" AS ((SELECT MAX("t2"."id") AS "min_id" FROM "challenge" AS "t2"))
    (SELECT "t3"."id" FROM "challenge" AS "t3" INNER JOIN "demo" ON ("t3"."id" = "demo"."min_id"))
)
```

While the CTE queries and the subquery themselves do get extra parentheses, the base query is not parenthesized leaving the `WITH` part bare and resulting in an invalid syntax.

The expected query would be:
```
SELECT "t1"."id", "t1"."challenge_id" FROM "attachment" AS "t1"
WHERE (
    "t1"."challenge_id" IN (
        WITH "demo" AS (SELECT MAX("t2"."id") AS "min_id" FROM "challenge" AS "t2")
        SELECT "t3"."id" FROM "challenge" AS "t3" INNER JOIN "demo" ON ("t3"."id" = "demo"."min_id"))
    )
)
```


This is probably a duplicate of #1809, but I'll leave it open for @coleifer to judge.

Thanks for reporting. Ive taken the description from this issue and added it as a comment to #1809. I'll deal with this as a single issue.
Issue: PooledPostgresqlExtDatabase is imported as None
Author: LearnedVector
State: closed
Created: 2019-01-27 17:10:55

```python
from playhouse.pool import PooledPostgresqlExtDatabase
print(PooledPostgresqlDatabase)  # returns None
```

digging into the code it looks like there is an import error causing this. any idea why?

https://github.com/coleifer/peewee/blob/master/playhouse/pool.py#L282



You need to install psycopg2.

awesome! thank you for the super fast response.
Issue: select from multiple subqueries
Author: michalchrzastek
State: closed
Created: 2019-01-27 16:03:07

Hi, trying to combine multiple subqueries into one, but can't see any guide for this, can you advise please.

 ```
SELECT q5.grpby,
    q5.cat AS category,
    q5.sumallmonths AS total,
    q4.avg12month AS avg12mnt,
   FROM ( 
	   SELECT tag_list."groupName" AS cat,
            tag_list."tagGroup" AS grpby,
            sum(transactions.trans_amnt) AS sumallmonths
           FROM tag_list
             LEFT JOIN transactions ON transactions.trans_cat = tag_list.id
          GROUP BY tag_list."tagGroup", tag_list."groupName"
   ) q5,
    ( SELECT tag_list."groupName" AS cat,
            round(sum(transactions.trans_amnt) / 12::numeric, 2) AS avg12month
           FROM tag_list
             LEFT JOIN transactions ON transactions.trans_cat = tag_list.id 
		AND transactions.trans_date >= date_trunc('month'::text, now()::date - '1 year'::interval) 
		AND transactions.trans_date <= (date_trunc('month'::text, now())::date - 1)
          GROUP BY tag_list."groupName"
		  ) q4
  WHERE q4.cat::text = q5.cat::text
  ORDER BY q5.grpby;
```

and my peewee attempt:
```

	def get_statistics(self):
		q5=tag_list.select(
tag_list.groupName.alias('tag')
,tag_list.tagGroup.alias('grpBy')
,fn.SUM(transactions.trans_amnt).alias('sumAllMonths'))
.join(transactions, on=(transactions.trans_cat == tag_list.id)).group_by(tag_list.tagGroup,tag_list.groupName)).order_by(tag_list.tagGroup)
		
q4=(tag_list.select(
tag_list.groupName.alias('tag')
,tag_list.tagGroup.alias('grpBy')
,fn.SUM(transactions.trans_amnt).alias('avg12Months'))
.join(transactions, on=(transactions.trans_cat == tag_list.id)).where(transactions.trans_date.between(first12MonthsAgo,lastOfPrevMonth)).group_by(tag_list.tagGroup,tag_list.groupName)).order_by(tag_list.tagGroup)

		query = q4 & q5
		return query
```

the webpage doesn't throw any error, and no results

Your code is completely unreadable. Please reduce to a minimal, clear example if you think there is a bug.

However this does not seem like a bug report, but rather like a question. In that case, the proper place to post is StackOverflow or you can try asking in the IRC channel (peewee on freenode), or on the mailing list.

A hint is that you should probably use `from_()` to list your multiple query sources. Or refactor to use common table expressions.

There are some examples that may help, here:

http://docs.peewee-orm.com/en/latest/peewee/query_examples.html#joins-and-subqueries

Ok, what about this:

```
Select q1.name,q1.countA,q2.countB
FROM
(select name, count()countA )q1,
(select name, count()countB )q2
```

How to combine 2 subqueries into 1?

```python

q1 = Foo.select(Foo.name, fn.COUNT(Foo.bar))
q2 = Foo.select(Foo.name, fn.COUNT(Foo.baz).alias('count'))

query = Foo.select(q1.c.name, q2.c.name, q2.c.count).from_(q1, q2)
```

Untested, but that's the jist of it.

Thanks, this helped.
Issue: Model.get() method fails silently
Author: arel
State: closed
Created: 2019-01-26 20:53:18

I came across a behavior with [`Model.get`](http://docs.peewee-orm.com/en/latest/peewee/api.html#Model.get) I found confusing and could be improved or at least better documented.

Calling `Model.get()` is a shorthand for "selecting with a limit of 1", which is clear. When you provide it filters, such as `.get(id=1442)`, things work as expected too. What was unexpected to me was that if you forget to specify the filter parameter by name such as `.get(1442)` the query does *not* fail. Rather, it returns an arbitrary result as if `.get(True)` or simply `.get()` were given. In my case, this led to a subtle bug because the two results were coincidentally very similar. For instance:

```
In [15]: Image.get(id=1442)                                                                                                                        
Out[15]: <Image: 1442>

In [16]: Image.get(1442)                                                                                                                           
Out[16]: <Image: 1433>
```

Is this the intended / desired behavior for some reason I'm missing? Or, would it be possible to raise an exception whenever the query expression is not an instance of type [Expression](http://docs.peewee-orm.com/en/latest/peewee/api.html#Expression)?

P.S. thanks for your work on this library!

This gets translated into:

```sql
select * from image where 1442
```

So the db interprets that, most likely, as simply `true` and returns the first row of this arbitrarily-large result set.

I'm aware of this issue, but not positive about the best way to fix it. Python is highly dynamic, and so is peewee. Adding validation to all SQL clauses (not just WHERE, which is in this example), seems prohibitively complicated.

Perhaps, though, the `.get()` method can be fixed up.

I've chosen to resolve this by supporting .get() with a single integer parameter, treating it as `.get(primary-key == value)`. So in the common case where one just passes an int, it will work as expected.

OK, thanks!
Issue: Mariadb 10.3.3+ change on values() function  
Author: altunyurt
State: closed
Created: 2019-01-21 21:14:08

This is more like a heads up! issue. 

https://mariadb.com/kb/en/library/values-value/ states that 
```
INSERT INTO t (a,b,c) VALUES (1,2,3),(4,5,6)
    ON DUPLICATE KEY UPDATE c=VALUES(a)+VALUES(b);
```
should be written as 
```
INSERT INTO t (a,b,c) VALUES (1,2,3),(4,5,6)
    ON DUPLICATE KEY UPDATE c=VALUE(a)+VALUE(b); <== this has changed 
```
starting from 10.3.3. 

Apparently peewee will fail at "on duplicate ... update ... values" queries on mariadb 10.3.3+ 
Judging from https://github.com/coleifer/peewee/blob/55f515be7012fcf8e33fe7f0b2726b7a36b729c0/tests/sql.py#L1287

Thanks for the heads-up. This looks like a pain.

* MySQL uses VALUES: https://dev.mysql.com/doc/refman/8.0/en/insert-on-duplicate.html
* MariaDB's changes do not appear to be backwards-compatible: https://mariadb.com/kb/en/library/values-value/
Issue: Feature Request: Restore `Query` __repr__/__str__ functionality
Author: ibushong
State: closed
Created: 2019-01-15 23:09:19

In peewee v2 it was convenient when debugging to print out the raw sql of a query with just `print query`, but  in v3 this just gives the generic `<peewee.ModelSelect object at 0x10abce150>`. For now I've just added it to my base peewee model, leveraging `sql()`, but I think this might be a nice feature to add back in v3?


That has been replaced by calling the `.sql()` method on a query.
Issue: When create_tables: OperationalError: near "AS": syntax error
Author: rafaleo
State: closed
Created: 2019-01-15 22:45:36

Hi. I've just installed peewee for the first time and try to execute some example. Based on your tutorial one (which works) I've created this simple code:
```
import peewee as pw

db = pw.SqliteDatabase('my_students.db')


class BaseModel(pw.Model):
    class Meta:
        database = db

class Student(BaseModel):
    name = pw.CharField(unique=True)
    points = pw.IdentityField()

db.connect()    
    db.create_tables([User, Student])
```
but it throws an error:
```
db.create_tables([User, Student])
Traceback (most recent call last):

  File "<ipython-input-83-9aeeec4d4064>", line 1, in <module>
    db.create_tables([User, Student])

  File "C:\Anaconda3\lib\site-packages\peewee.py", line 2789, in create_tables
    model.create_table(**options)

  File "C:\Anaconda3\lib\site-packages\peewee.py", line 5670, in create_table
    cls._schema.create_all(safe, **options)

  File "C:\Anaconda3\lib\site-packages\peewee.py", line 4919, in create_all
    self.create_table(safe, **table_options)

  File "C:\Anaconda3\lib\site-packages\peewee.py", line 4805, in create_table
    self.database.execute(self._create_table(safe=safe, **options))

  File "C:\Anaconda3\lib\site-packages\peewee.py", line 2653, in execute
    return self.execute_sql(sql, params, commit=commit)

  File "C:\Anaconda3\lib\site-packages\peewee.py", line 2647, in execute_sql
    self.commit()

  File "C:\Anaconda3\lib\site-packages\peewee.py", line 2438, in __exit__
    reraise(new_type, new_type(*exc_args), traceback)

  File "C:\Anaconda3\lib\site-packages\peewee.py", line 177, in reraise
    raise value.with_traceback(tb)

  File "C:\Anaconda3\lib\site-packages\peewee.py", line 2640, in execute_sql
    cursor.execute(sql, params or ())

OperationalError: near "AS": syntax error
```
Can't find any key difference between my code and tutorial. Please give me a hint if it's not a bug.


This class is there - it's not the case. It doesn't work even if I pass only [Student] table.
```
class User(BaseModel):
    username = pw.CharField(unique=True)
```

What tutorial are you referring to? Please share the link.

The problem is your usage of `IdentityField`. You are using a `SqliteDatabase` in your example, but the documentation for `IdentityField` *clearly* states that this field is for postgresql 10.0 and newer ONLY.

http://docs.peewee-orm.com/en/latest/peewee/api.html#IdentityField

You should use an `IntegerField` for storing a point value.

Ok. I've found that `IdentityField` is somehow a wrong wolf, I've changed to `IntegerField` and it's all right. If it's not an issue just ignore this.