akumoth/peewee-issues huggingface dataset:
Issue: DateTimeField get <unicode> instead of <datetime> in special case 
Author: icewisb
State: closed
Created: 2017-10-11 07:46:16

With use of fn.MAX and join 3 tables and group_by, I can't get DateTimeField with type <datetime> correctly.

```python
from peewee import JOIN, fn
from server.models import SaoShip, SaoMstr, SaoStyle

store = '001'
company = '10011'

m = SaoMstr.alias()
t = SaoStyle.alias()
s = SaoShip.alias()

query = m.select(fn.MIN(s.ShipDate).alias('min_end_date')) \
    .join(s, on=(s.OrderNo == m.OrderNo)) \
    .join(t, on=(t.OrderNo == m.OrderNo)) \
    .where(m.ExtTerm7 == store, m.Customer == company)\
    .group_by(t.OrderNo, t.Style)

record = query.get()

print type(record.min_end_date)
```

Terminal output :
```shell
<type 'unicode'>
```

What version of Peewee? What database driver?

Closing due to inactivity.

My peewee is 2.10.2 in python2, this case only  appear when I use three tables. It disappeared when I leave two talbes

Database driver is mysqldb.

That's rather odd. Is there a reason you're using the call to `.alias()` all over the place instead of using the models themselves? What happens if you remove the calls to `alias()`?

Ok,I will try it tomorrow. and I have an anther issue about BlobField , I use this field to store Binary img object successfully, but when I get this record use peewee why I get unicode object instead of binary object.


```
query = SaoMstr.select(fn.MIN(SaoShip.ShipDate).alias('min_date'), SaoStyle.Style) \
    .join(SaoShip, on=(SaoShip.OrderNo == SaoMstr.OrderNo)).join(SaoStyle, on=(SaoStyle.OrderNo == SaoMstr.OrderNo)) \
    .where(SaoMstr.ExtTerm7 == '001', SaoMstr.Customer == '10011').group_by(SaoStyle.Style)

In [28]: type(query.get().min_date)
Out[28]: unicode
```
When I delete ```SaoStyle.Style```:
```query = SaoMstr.select(fn.MIN(SaoShip.ShipDate).alias('min_date')) \
    .join(SaoShip, on=(SaoShip.OrderNo == SaoMstr.OrderNo)) \
    .switch(SaoMstr).join(SaoStyle, on=(SaoStyle.OrderNo == SaoMstr.OrderNo).alias('st')) \
    .where(SaoMstr.ExtTerm7 == '001', SaoMstr.Customer == '10011').group_by(SaoStyle.Style)

In [34]: type(query.get().min_date)
Out[34]: datetime.datetime
```
And the following thing is very strange.
```
query = SaoShip.select(fn.MIN(SaoShip.ShipDate).alias('min_date'), SaoStyle.Style).join(SaoStyle, on=(SaoStyle.OrderNo == SaoShip.OrderNo)).where(SaoShip.OrderNo == '0001000000001').group_by(SaoStyle.Style)

In [47]: type(query.get().min_date)
Out[47]: unicode

```
What cause it ?





Can you create a minimal example I can use to try and replicate this issue?
Issue: AttributeError: 'NoneType' object has no attribute 'sqlite_version_info'
Author: avish-3pg
State: closed
Created: 2017-10-08 13:29:34

@coleifer Still facing the sqlite3 issue, even though we are using peewee==2.10.1. Please advice at the earliest. Thank you.

Error Stack Trace - 
'NoneType' object has no attribute 'sqlite_version_info': AttributeError
Traceback (most recent call last):
File "/var/task/handler.py", line 511, in lambda_handler
return LambdaHandler.lambda_handler(event, context)
File "/var/task/handler.py", line 242, in lambda_handler
return handler.handler(event, context)
File "/var/task/handler.py", line 335, in handler
app_function = self.import_module_and_get_function(whole_function)
File "/var/task/handler.py", line 233, in import_module_and_get_function
app_module = importlib.import_module(module)
File "/var/lang/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 978, in _gcd_import
File "<frozen importlib._bootstrap>", line 961, in _find_and_load
File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
File "/var/task/schedular.py", line 16, in <module>
import models_postgres as models
File "/var/task/models_postgres.py", line 4, in <module>
from playhouse.db_url import connect
File "/var/task/playhouse/db_url.py", line 118, in <module>
from playhouse.berkeleydb import BerkeleyDatabase
File "/var/task/playhouse/berkeleydb.py", line 10, in <module>
sqlite3_lib_version = sqlite3.sqlite_version_info
AttributeError: 'NoneType' object has no attribute 'sqlite_version_info'


Try 8ba62478f7ce4fc3890fad0d1c42e851a382bdd4 ?

Closing due to inactivity.
Issue: Database URL extension for retrieving endpoint information from AWS RDS
Author: mguillau
State: closed
Created: 2017-10-06 11:50:48

This pull-request adds an extension to playhouse's database URL mechanism to handle two new URL schemes, namely rds:// and rdsro:// (for rds "read-only"). These schemes simplify the use of AWS' Relational Database Service (RDS).

Through boto3 (the main python SDK for AWS), RDS can provide the host endpoint (rw or ro), its port, the master username and the database engine type (mysql, postgresql, aurora and mariadb supported for far in this patch) to any authorised AWS users, from just the database id (a case-insensitive string). With this information, it is then easy to build the actual database URL for peewee/playhouse to use for connecting.

For instance, one could provide the simple url "rds://test_db_user:test_db_pass@test_rds_db_id" which would automatically resolve to "mysql://test_db_user:test_db_pass@test_rds_db.ceqrivboeuyrbj.us-west-2.rds.amazonaws.com:5555/testdb" (assuming the RDS instance is mysql-compatible, listening on port 5555, etc.). The benefit is that the URL does not need to be changed as one manages its RDS database (changing the port, replacing hosts, adding replication and growing the fleet, etc.)

The main idea of the patch is to provide the function that converts a parsed rds:// or rdsro:// url into the corresponding parsed mysql:// or postgresql:// URL (dynamically determined depending on the engine type). The commit also provides a mechanism to add more such functions to resolve parsed URLs into new parsed URLs. Potential applications for this mechanism include reading database URLs or passwords from environment variables, configuration files, else other secret-sharing services (those applications are left for future pull requests).

Looking forward to comments and suggestions!

Because I'm not interested in maintaining this functionality, I'd suggest packaging this as a standalone project.

It seems like the schema resolver change would be necessary to build this as an independent package.

Thanks for the comments. I can easily re-submit a patch containing only the schema resolution part in db_url. Indeed, this would make it much easier to create standalone extensions as suggested. Would such a pull-request be of enough interest to you?
Issue: More advanced index creation
Author: Necoro
State: closed
Created: 2017-10-03 19:42:23

Currently one can only define indexes using a tuple consisting of column names and uniqueness. With this construct it is not possible to create functional or partial indexes, for instance.

While ways exist, it is a) what way is to be preferred and b) most often the declaration of that index will be somewhere else than all the other indexes of a table.

I therefore suggest add an `Index` class that can be used in lieu of the tuple and which encapsulates the creation of index-related data (name, query). So, whenever one needs special constructs that peewee does not offer, a simple subclass is sufficient.

I love it!

When you're declaring an index inside the `Meta` attribute, the Python interpreter hasn't constructed the actual model class yet, so you can't do neat things like this:

```python
class Article(Model):
    name = TextField()
    timestamp = TimestampField()
    status = IntegerField(default=1)
    flags = IntegerField(default=0)

    class Meta:
        indexes = (
            Index(fn.SUBSTR(Article.name, 1, 1), Article.timestamp.desc()).where(Article.status == 1),
        )
```

So I think an alternative would be to simply allow `SQL()` objects to be included alongside the tuples? You'd have to specify the full SQL for the index, but at least that guarantees everything is supported. WDYT?

Added in 3.0a
Issue: Cannot specify schema with PostgresqlMigrator
Author: irlabs
State: closed
Created: 2017-09-21 14:44:19

I'm trying to use `playhouse.migrate` to modify the database according to the model changes, but it seems I cannot specify the PostgreSQL schema of the table that I need to alter.

My model looks like this:

```python
db = PostgresqlDatabase(...)

class MyTable(Model):
    title = CharField()
    class Meta:
        database = db 
        db_table = 'table_name'
        schema = 'my_schema'

```

But if I run migration code like the following:

```python
migrator = PostgresqlMigrator(db)
new_field = CharField(default="")
migrate(
    migrator.add_column(MyTable._meta.db_table, 'new_field', new_field),
)
```

, I get this error:

```
peewee.ProgrammingError: relation "table_name" does not exist
```

How can I tell the migrator that it should alter the table `table_name ` in the schema `my_schema `?

You're right. You may need to, in the meantime, run something like `db.execute_sql('set search_path to my_schema')` until I can get a fix.

Did you try the above suggestion, by chance?

> Did you try the above suggestion, by chance?

No, I used the psql command line instead.

Probably related, but I'm getting the same error using reflection on a Postgresql DB with a schema. `generate_models()` works fine but making a query on any of the generated models fails. If I do `db.execute_sql('set search_path to targetdb'`) the queries do work.

@albireox -- check out febcb495de5f21bc19c8834af78522fbfdb6c994

Yep, that seems to fix the problem.

Rather than modify all the APIs to support a schema parameter, I added a method which you can use to set the search path for subsequent operations. e.g.

```python

title_field = TextField(default='')
migrate(
    migrator.set_search_path('my_schema'),
    migrator.add_column('some_table', 'title', title_field),
)
```
Issue: Allow releasing of stale connections back to pool.
Author: sebastienvercammen
State: closed
Created: 2017-09-19 16:00:01

Fine-grained control over the connection pool in non-web contexts can become more frustrating with the lightweight implementation because it's not implemented in peewee's core (e.g. after finishing a query, the connection isn't auto-released back to the pool), and in some structures context managers or decorators aren't desirable (requiring global access to the database's instance).

Adding this (very simple) option allows us to release connections back to the pool when they are stale rather than closing them entirely. This lets the user decide on how to manage what peewee considers stale connections.

There is one downside: the connection pool doesn't/can't know whether the query has finished so it's possible it will release busy connections back to the pool.

I hope this PR can serve as a discussion board for a better solution, because calling `Model._meta.database.close()` after every query or needing global access to the database's instance is not ideal.

I'm not sure how a connection would still be busy, since peewee will block until the query is finished. Furthermore, connections aren't returned to the pool until you call `close()` on them.

If you don't wish for connections to be marked stale, you can also modify the `stale_timeout` parameter (which, by default, is not set).

@coleifer We've restructured the original application that brought us to this use case, but I submitted the PR in case there's a use for releasing connections to the pool rather than closing them.

You're correct that peewee blocks for queries, but I was thinking about what would happen if a different thread went through the connections and found one that passed `stale_timeout` while its original thread was still waiting for a response.

The connection wouldn't be available to another thread, as it would be "checked out" by the connection that was using it. There's a lock around connect/close, so no race condition either.

Connections are released to the pool to be re-used when you call close() in the application. The pool itself will then close the connection if it is unusable or stale (if you've set up a stale timeout).

@coleifer I seem to be overlooking the locks around connect/close in `PooledDatabase`, could you link me to it?

They're in peewee's `Database` object.

@coleifer That makes an awful lot more sense, thanks for clarifying. If you see no use for the PR (which no longer fits our own use case), feel free to close it.

With peewee blocking and waiting for responses, there might be a use case for someone that wants to use `stale_timeout` as a soft "release to pool" timeout that would assume the author didn't explicitly call close.

Glad to help
Issue: order_by didn't work
Author: EverCurse
State: closed
Created: 2017-09-15 11:00:42

when i use peewee to order my result like this:
`us = Users.select().order_by(Users.id.desc())`
i find desc didn't work, so i use 
`us = Users.select().order_by(Users.id.asc())`
the result is same as desc , so why desc didn;t work? 
thanks 

it is my fault ,my code logic is error , thanks for peewee
Issue: How to force not to get primary key on insert action
Author: soyking
State: closed
Created: 2017-09-15 08:16:52

I use peewee as PostgreSQL orm. As I saw on [InsertQuery.execute](https://github.com/coleifer/peewee/blob/81091b87c8c4b4d3dc65fc63b985629cd70dfd3a/peewee.py#L3584), when `is_insert_returning` is set, peewee will use `RETURNING` to get primary key. Otherwise it will call `last_insert_id` which is implemented by `SELECT CURRVAL` in PostgreSQL. 

In my case, I use partition function on Postgres. So I will `RETURN NULL` when inserting a row, which means `RETURNING` won't work. And `SELECT CURRVAL` will fail as well.

So I open this issue to make sure that whether there is a way to force not to get primary key after insert action. Thanks!

So...you bring up a good point. In peewee 2.x it's not really possible using the existing APIs.

In the 3.x branch, you can now write:

```python
SomeModel.insert({SomeModel.field: 'value'}).returning(None)
```

@coleifer 

Thanks for your response. It helps a lot.

ps. Is there any way to run raw query in 3.0 ? I didn't find the API actually.

There is no more `RawQuery` in 3.0a. You can use `Database.execute_sql()` instead, which returns a cursor object.
Issue: Unnecessary parentheses in the produced SQL
Author: tuukkamustonen
State: closed
Created: 2017-09-14 11:40:08

```python
Foo.select().where((Foo.id == 1) | (Foo.id == 2) | (Foo.id == 3) | (Foo.id == 4))
```

Produces SQL like:

```sql
... WHERE ((((\"t1\".\"id\" = 1) OR (\"t1\".\"id\" = 2)) OR (\"t1\".\"id\" = 3)) OR (\"t1\".\"id\" = 4)) ...
```

And so on. I don't know if the extra parentheses have impact on performance, but I would think so?

Using `2.10.0` with postgres.

The parentheses are semantically correct, though to a human they look extraneous. It's completely acceptable.
Issue: Delete() based on a dictionary instead of an Expression.
Author: thxv3n0lvl
State: closed
Created: 2017-09-12 12:48:17

I have this dictionary
`table_fields = {'entity_id': 9998777}` 

I can pass it to my own implementation of **get_or_create** by doing something like:
```    
        def get_or_create(table_fields):
        try:
            return AdNetworkModel.get(**table_fields).id
        except AdNetworkModel.DoesNotExist:
            return AdNetworkModel.create(**table_fields).id
```
My model looks like:
```
class AdNetworkModel(BaseModel):
    entity_id = peewee.CharField()

    class Meta:
        db_table = 'AD_NETWORK'
```

Until now all good, the get_or_create method works fine, but if I want to make a `delete()` it doesn't work, seems that I have to use an Expression which I wouldn't like to because I want to create my own dynamic delete based on a dictionary. Is this possible? 

```
  def delete(table_fields):
        AdNetworkModel.delete(**table_fields).execute()
```

The error that this code throws is:

```
{TypeError}delete() got an unexpected keyword argument 'entity_id'
```

You can use:

```python
AdNetworkModel.delete().filter(**table_fields).execute()
```

If you're working on serving advertisements, though, I probably shouldn't help you :)

Thanks sir !
Issue: Inserting a list of IntEnum members into ArrayField(IntegerField) fails
Author: DragoonAethis
State: closed
Created: 2017-09-04 14:44:35

Trying to use IntEnum members as values in `playhouse.postgres_ext.ArrayField(IntegerField)` will fail, since peewee wants to insert names of those values (for instance `"SomeEnum.example"`) instead of the actual values (eg. `1`).

On PostgreSQL 9.6.4, peewee 2.10.1 and Python 3.6.2, this code:

```python
from enum import IntEnum

import peewee as pw
from playhouse.postgres_ext import *

# Behaves the same way in PostgresqlDatabase too.
db = PostgresqlExtDatabase("exploding-db",
	user="user", password="nope", host="localhost")

class Role(IntEnum):
	can_sign_in = 0
	view_posts = 1
	edit_posts = 2
	delete_posts = 3
	admin = 1000

class User(pw.Model):
	class Meta:
		database = db

	login = pw.TextField(unique=True)
	password = pw.TextField()  # PasswordField IRL, but let's keep it simple
	roles = ArrayField(pw.IntegerField)

db.drop_tables([User], safe=True)
db.create_tables([User])
user = User(login="admin", password="test", roles=[Role.admin])
user.save()  # Boom!
```

...fails with:

```
Traceback (most recent call last):
  File "/somewhere/in/the/universe/peewee-boom.py", line 30, in <module>
    user.save()  # Boom!
  File "/usr/lib/python3.6/site-packages/peewee.py", line 5108, in save
    pk_from_cursor = self.insert(**field_dict).execute()
  File "/usr/lib/python3.6/site-packages/peewee.py", line 3526, in execute
    cursor = self._execute()
  File "/usr/lib/python3.6/site-packages/peewee.py", line 2912, in _execute
    return self.database.execute_sql(sql, params, self.require_commit)
  File "/usr/lib/python3.6/site-packages/playhouse/postgres_ext.py", line 390, in execute_sql
    self.commit()
  File "/usr/lib/python3.6/site-packages/peewee.py", line 3598, in __exit__
    reraise(new_type, new_type(*exc_args), traceback)
  File "/usr/lib/python3.6/site-packages/peewee.py", line 135, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3.6/site-packages/playhouse/postgres_ext.py", line 383, in execute_sql
    cursor.execute(sql, params or ())
peewee.ProgrammingError: missing FROM-clause entry for table "role"
LINE 1: ...assword", "roles") VALUES ('admin', 'test', ARRAY[Role.admin...
                                                             ^
```

Since [IntEnum](https://docs.python.org/3/library/enum.html#intenum) members are subclasses of ints, it should be possible to assign those members directly. Many other Python mechanisms treat them much like integers (so, say, `Role.admin == 1000` works as you'd expect), yet it doesn't in there.

The workaround is to use `MyNeatEnum.member.value` instead of `MyNeatEnum.member`, but that gets a little bit repetitive. It'd be nice if this was handled automatically.

peewee is handing the `IntEnum` members directly to psycopg2, so my sense is that the issue lies there. An alternative would be to subclass ArrayField and override the `db_value()` method, coercing each list item into an `int()`.
Issue: add truncate_table to doc
Author: toshitanian
State: closed
Created: 2017-09-01 04:02:07

truncate_table is implemented on peewee, but not in doc. i hope it is written in doc.

https://github.com/coleifer/peewee/pull/862/files


That's true -- I will look at adding similar functionality to the 3.0a branch and will document it there, as Peewee will be moving to the 3.0 branch soon.

Well, actually, I think I'm going to omit it from the 3.x branch for now, so will leave as-is. But thank you for letting me know.
Issue: pwiz creates incorrect code for composite foreign keys
Author: fredrikchabot
State: closed
Created: 2017-08-29 21:04:01

Given this (PostgreSQL):
```sql
CREATE TABLE tablea (
    keyparta character varying(4) NOT NULL,
    keypartb integer NOT NULL,
    whatever character varying(20)
);

CREATE TABLE tableb (
    keyparta character varying(4) NOT NULL,
    keypartb integer NOT NULL,
    keypartc character varying(16) NOT NULL,
    whatever character varying(12)
);

ALTER TABLE tablea ADD CONSTRAINT tablea_primary PRIMARY KEY (keyparta, keypartb);
ALTER TABLE tableb ADD CONSTRAINT tableb_primary PRIMARY KEY (keyparta, keypartb, keypartc);
ALTER TABLE tableb ADD CONSTRAINT tablea_keyparta_keypartb FOREIGN KEY (keyparta, keypartb) REFERENCES tablea(keyparta, keypartb);
```
pwiz creates a correct Tablea class, and a nonfunctioning Tableb class.
```python
class Tableb(BaseModel):
  keyparta = ForeignKeyField(db_column='keyparta', rel_model=Tablea, related_name='tablea_keyparta_set', to_field='keypartb')
  ....
```
I wouldn't know how to express this schema either but having part of a foreign key pointing at a different part of it in the target table definitely doesn't work.


Fredrik

Peewee does not support foreign keys to composite primary keys using any defined APIs.

Edited issue to add syntax highlighting and proper formatting.