akumoth/peewee-issues huggingface dataset:
Issue: peewee, flask and python3.6 
Author: noxizcm
State: closed
Created: 2017-08-29 19:04:13

`sqlite3_lib_version = sqlite3.sqlite_version_info
AttributeError: 'NoneType' object has no attribute 'sqlite_version_info'`
from playhouse/berkeleydb.py
how to fix it?

You need the standard library sqlite3 installed at the very least.

Is there a way to avoid using sqlite3 all together?
Or is sqlite3 a peer dependency of peewee?

I'm encountering the same error on AWS Lambda. In my `models.py` I have the following defined:

```
from peewee import *
from playhouse.db_url import connect

db = connect(os.environ.get('DATABASE_URL'))

class BaseModel(Model):
    class Meta:
        database = db

# more models
# ...
```

And when the lambda is executed, I get the following:

```
START RequestId: ... Version: $LATEST
module initialization error: 'NoneType' object has no attribute 'sqlite_version_info'
```

What version of Peewee are you using? Could you share the full traceback? It seems that this shouldn't occur in the latest 2.x code:

https://github.com/coleifer/peewee/blob/master/playhouse/sqlite_ext.py#L70-L73

Still facing the sqlite3 issue, even though we are using peewee==2.10.1. Please advice at the earliest. Thank you.

Error Stack Trace -
'NoneType' object has no attribute 'sqlite_version_info': AttributeError
Traceback (most recent call last):
File "/var/task/handler.py", line 511, in lambda_handler
return LambdaHandler.lambda_handler(event, context)
File "/var/task/handler.py", line 242, in lambda_handler
return handler.handler(event, context)
File "/var/task/handler.py", line 335, in handler
app_function = self.import_module_and_get_function(whole_function)
File "/var/task/handler.py", line 233, in import_module_and_get_function
app_module = importlib.import_module(module)
File "/var/lang/lib/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 978, in _gcd_import
File "", line 961, in _find_and_load
File "", line 950, in _find_and_load_unlocked
File "", line 655, in _load_unlocked
File "", line 678, in exec_module
File "", line 205, in _call_with_frames_removed
File "/var/task/schedular.py", line 16, in 
import models_postgres as models
File "/var/task/models_postgres.py", line 4, in 
from playhouse.db_url import connect
File "/var/task/playhouse/db_url.py", line 118, in 
from playhouse.berkeleydb import BerkeleyDatabase
File "/var/task/playhouse/berkeleydb.py", line 10, in 
sqlite3_lib_version = sqlite3.sqlite_version_info
AttributeError: 'NoneType' object has no attribute 'sqlite_version_info'

8ba62478f7ce4fc3890fad0d1c42e851a382bdd4
Issue: Add --ssl option to pwiz
Author: mgalvey
State: closed
Created: 2017-08-26 22:30:40

Refs #1336 

Tested locally with both supported mysql drivers and didn't run into any problems. DID NOT test that psycopg tries SSL by default, but the docs for the client library that psycopg uses say that it does: https://www.postgresql.org/docs/9.1/static/libpq-ssl.html. From that page:

> The default value for sslmode is prefer.

This enables only the most simple ssl mode (with no CA verification), but it at least gets you around the problem of a server configured only to allow secure connections.

Going to pass on this, but thank you.

Is there anything that would make this or a feature like it suitable for inclusion in pwiz, or is this use case just something you're not interested in supporting?

In 5...6 years you're the only person who's requested it :)

@coleifer I guess I should say that we really need this. I'm not sure how we can use peewee in a anything with even mildly sensitive data without SSL. Basically, it's for the exact reason that @mgalvey was saying.

Is this something you'd consider adding in for MySQL?
Issue: Filtering by TimestampField in Postgres
Author: Alexeyisme
State: closed
Created: 2017-08-26 13:48:25

Hello!

I'm trying to filter queryset I got from remote Postgres DB by TimestampField, and can't find out the timestamp format to cast.
I tried:
`today_events = events.select().where(fed_events.created_at > '2017-08-25 00:00:00')`
`today_events = events.filter(fed_events.created_at > '2017-08-25 00:00:00')`
`today_events = events.filter(fed_events.created_at > 1503604800.0)`
Also tried datetimes.

I get errors like "You may need to cast the value explicitly".
Do I miss something?

Best Regards,
Alexey

Solved. [Docs](http://docs.peewee-orm.com/en/latest/peewee/api.html?highlight=trunc#Database.truncate_date).
```
today_events = events.select().where(fn.date_trunc('day', fed_events.created_at) == datetime.date.today())
```
Issue: full text postgres search crashes with more than one word
Author: wilsonrocks
State: closed
Created: 2017-08-25 08:32:35

This seems to be a postgres thing. I wrote some code to convert "multiple search terms! with punctuation." to "multiple & search & terms & with & punctuation" which doesn't crash it. Would this be useful in peewee, perhaps in the Match function?

Peewee probably won't ever handle escaping search queries -- that should probably be done by the application as the needs may vary from app to app.
Issue: Shouldn't ensure_join unfold aliases?
Author: insolite
State: closed
Created: 2017-08-24 09:29:38

```python
class Author(peewee.Model):
    name = peewee.CharField()

class Book(peewee.Model):
    name = peewee.CharField()
    author = peewee.ForeignKeyField(Author)

query = (Book
         .select(Book.name)
         .join(Author.alias(), peewee.JOIN_LEFT_OUTER)
         .filter(author__name='someone'))
print(query.sql()[0])
```

`ensure_join` call inside `filter` generates additional join on `author` table because of the `Author.alias()`. The compiled query looks like:
```sql
SELECT "t1"."name"
FROM "book" AS t1
LEFT OUTER JOIN "author" AS t2 ON ("t1"."author_id" = "t2"."id")
INNER JOIN "author" AS t3 ON ("t1"."author_id" = "t3"."id")
WHERE ("t3"."name" = ?)
```

So shouldn't `Query.ensure_join` consider model and it's alias as the same model and do not generate additional join (`t3`) or I'm missing something and it is necessary?

P. S.
I know that it can be solved using `where` (see code below), but I've provided simplified test case. In my real code I have to use `filter` anyway. So just interesting if it's going to be changed or I must write my own `filter`.
```python
author_alias = Author.alias()
query = (Book
         .select(Book.name)
         .join(author_alias, peewee.JOIN_LEFT_OUTER)
         .where(author_alias.name == 'someone'))
```


Just realized that `filter` is actually deprecated, so I guess there no plans for it. Anyway `ensure_join` is also used inside `SelectQuery.annotate` so maybe it still makes sense.

Hi, interesting I hadn't thought about this issue. You're correct as far as filter being deprecated, but it's still probably worth resolving this in some fashion. I would point out that filter supports a subset of the functionality of where, so if filter works there is a solution using where.

06860a0a7619ffd3d0f80cbb81ce7e66ab632cb5 in the 3.0a branch fixes this issue. It also supports the use of the alias assigned to a model as a portion of the lookup:

```python
# works as expected in 3.0a:
UserAlias = User.alias('ua')
query = (Tweet
         .select(Tweet.content, UserAlias.username)
         .join(UserAlias)
         .filter(ua__username='foo')

for tweet in query:
    print tweet.user.username, tweet.content
```
Issue: Nested Join
Author: juntatalor
State: closed
Created: 2017-08-22 21:15:37

Cannot find a way to do nested join, like:

SELECT *
FROM t1
    LEFT OUTER JOIN t2
         INNER JOIN t3 ON (t2.x_id = t3.id)
    ON (t1.y_id = t2.id)

This query is much faster than two left joins for my particular task (PostgreSQL). Subquery looks ugly.

What?! This is some next-level shit!

Can you explain what, semantically, you're doing in the above query? Like...use tables with familiar names/concepts, e.g.

```sql
SELECT *
FROM users AS t1
LEFT OUTER JOIN notes AS t2
INNER JOIN note_tags AS t3
    ON (t2.id = t3.note_id) ON (t1.id = t2.user_id)
```

I'm not even sure if the above makes sense...can you explain?

https://stackoverflow.com/questions/10334608/nested-inner-join-vs-inner-join-vs-where-correctness-performance-clarity-for

and

http://michaeljswart.com/2012/09/when-i-use-nested-joins/

The key idea is that peewee can do either 2 left joins on table **t1** or left join and inner join on table **t1**. The second variant is incorrect, because some rows are truncated (null values).
The first one is bad because 2 left joins cost much.
Doing nested join performs inner join on t2 -> t3 and then left join with the result (less rows for nested loop operation is done faster)
Finally i resolved this issue with doing a nested select query (despite it looks ugly).
Thanks for your ORM, it works great. Not every sql feature can be simply done with query builders, it's okay.

Thanks for the explanation, that makes sense to me...interesting. I hadn't heard of doing such a thing, but yeah, makes sense I guess. I'm surprised that Postgres' query planner doesn't automatically optimize the naive LEFT/LEFT query, unless there's some semantic difference I'm not aware of...I suppose that there could be more result rows with the LEFT/LEFT than with the LEFT->INNER, so they're not exactly equivalent?

One approach that may work using the existing APIs is to use a subquery, e.g.

```python
class Employee(Model):
    name = TextField()

class OrgChart(Model):
    employee = ForeignKeyField(Employee, backref='bosses')
    boss = ForeignKeyField(Employee, backref='reports')
    class Meta:
        primary_key = CompositeKey('employee', 'boss')

# Get employees and their boss.
Boss = Employee.alias('boss')
subq = (OrgChart
        .select(Boss.name.alias('boss'), OrgChart.employee.alias('emp'))
        .join(Boss, on=(OrgChart.boss == Boss.id))
        .alias('subq'))
query = (Employee
         .select(Employee.name, subq.c.boss)
         .join(subq, JOIN.LEFT_OUTER,
               on=(Employee.id == subq.c.emp))
         .order_by(Employee.name))

for empname, bossname in query.tuples():
    print empname, '=>', bossname
```

Given that it is possible to accomplish the same with existing APIs (albeit using a different construction) I think I'll pass on trying to implement this... I'm not sure of what a good API would even be.
Issue: primary_key=False is not inheritable
Author: maaaks
State: closed
Created: 2017-08-22 12:13:03

[This table](http://docs.peewee-orm.com/en/latest/peewee/models.html#model-options-and-table-metadata) says that `primary_key` is an inheritable meta field. But it doesn't seem so, at least not when I set `primary_key=False` on a parent model and expect a child class to get the same.

If it's a bug, I think it should be fixed.
If it's supposed to be this way, I think it's worth mentioning in documentation.

This is working correctly in 3.0a -- I'm not inclined to change it in the 2.x branch, despite the documentation being somewhat confusing.

023037adf3d87127969c71bc95e6e6e070590873
Issue: How much efficiency has been raised?
Author: yanghg-basefx
State: closed
Created: 2017-08-22 02:09:49

I compiled _speedups.pyx at centos7, then try to fetch a whole table from MySQL database. I'm sure the speedups module was imported successfully, but it spent the same time as normal module. Does it work?

So, the peewee 2.x speedups module should help noticeably with:

* Iterating over large result sets for simple selects (no joins) or when the return row type is `dict` or `tuple` (`.dicts()` or `.tuples()`).
* Resolving dependencies among a large number of models.

It's a pretty narrow range, and honestly, the gains are not that huge. With SQLite in-memory database containing 100K rows, iterating through them all yields:

- select models 5.412
- select dicts 4.114
- select tuples 4.231

And with speedups:

- select models 5.229
- select dicts 3.743
- select tuples 3.718

In the 3.x branch, there is no longer a "speedups" module simply because I didn't feel the maintenance burden of the additional code justified the tiny improvement. It was more of an experiment than anything else, I suppose.

Timings for 3.0a, just in case you were curious, are comparable to `master`:

- select models 5.751
- select dicts 4.268
- select tuples 4.271

Here's the quick little script I put together to generate these numbers: https://gist.github.com/coleifer/7050c85d9e48dbe1307d7d648f7628a7

Ah, I got it, thanks.
Issue: How to integrate peewee and Flask-login
Author: noxizcm
State: closed
Created: 2017-08-19 12:09:11

D!
i am trying to use peewee and flask-login but wen create and user loader i have this error 
`    @login_manager.user_loader()
TypeError: user_loader() missing 1 required positional argument: 'callback'`
i knew that peewee have get_id() method but how to use it?

I think you have to read that https://flask-login.readthedocs.io/en/latest/#flask_login.LoginManager.user_loader
And implement methods in your User-model that were discussed flask-login.

Sorry, please direct this question to a more appropriate forum.
Issue: Is it possible to get a count of a grouped field?
Author: iancullinane
State: closed
Created: 2017-08-18 18:14:31

I have the following MySQL query:

`select Title, Field, count(*) as 'count' from model group by Title;`

Which returns:

`{ Title: 'Something', Field: 'Something Else', count: '5' } `

Where count is the number of occurrences of Title. I have been banging my head figuring out how to get this through peewee with no luck. Is this possible? The closest I have come sees the `Title` return overwrite the `Title` field so instead of 'Something' I get the count.

Well, you need to group by both title and field if you're selecting both (unless you use mysql, which allows shenanigans like this).

If you have a lot of people, this prints out the surnames and the count associated with each, e.g.:

```python
query = Person.select(Person.last_name, fn.COUNT(Person.id).alias('ct')).group_by(Person.last_name)
for person in query:
    print person.last_name, person.ct
```
Issue: Why use exec?
Author: thomwiggers
State: closed
Created: 2017-08-16 21:11:25

On this line you're using `exec` to define `reraise`. I don't understand why you just don't use `def` directly. Would you mind clearing this up for me?

https://github.com/coleifer/peewee/blob/7beab7c60ca32477fb812eb768adc4ac458e5a57/peewee.py#L144

because on python 3 `raise tp, value, tb` will raise `SyntaxError`, use `exec` to skip syntax check.

Ah, TIL. Thanks
Issue: Database.is_closed() returns incorrect value sometimes
Author: skewty
State: closed
Created: 2017-08-16 17:02:23

I sometimes get this:  `psycopg2.InterfaceError: connection already closed` and can't seem to track it down.  But I was able to determine that Database.is_closed() returns an incorrect value sometimes.

I looked in peewee.py at the close() method of the Database class and would like to propose a small improvement:

Currently:
```
with self.exception_wrapper:
    self._close(self._local.conn)
    self._local.closed = True  # this line is inside the with
```
Proposed:
```
with self.exception_wrapper:
    self._close(self._local.conn)
self._local.closed = True  # move it outside the with (should it go back 1 or 2 levels??)
```

How would moving that outside the block change things?

if I am understanding the code correctly:

Currently if the call to `self._close(self._local.conn)` raises an Exception then `with self.exception_wrapper` will `__exit__` and `self._local.closed = True` will never be called.

By moving `self._local.closed = True` out of the `with` it should always get executed.

The exception_wrapper simply wraps the exceptions raised by the database driver to a "peewee" equivalent exception. It doesn't suppress the exception. Thus you can catch a peewee.OperationalError rather than needing to catch both psycopg2 / sqlite3 if you were so inclined, e.g.

Okay, then perhaps something like this:

```python
def close(self):
    with self._conn_lock:
        if self.deferred:
            raise Exception('Error, database not properly initialized '
                            'before closing connection')
        try:
            with self.exception_wrapper:
                self._close(self._local.conn)
        finally:
            self._local.closed = True
```

Are you intending to catch the `InterfaceError` yourself and ignore it?

b06a9b0
Issue: model_to_dict not working with generic foreign keys
Author: hishamkaram
State: closed
Created: 2017-08-16 10:13:26

I am trying to include tags in object dict to convert this data to json then use this data in api how can i achieve that?(if models_to_dict can include tags in dict)
select query: 

`queryset = model.select()`

Tag Model:
```
class Tag(BaseDateTime):
    tag = CharField()
    object_type = CharField(null=True)
    object_id = IntegerField(null=True)
    object = GFKField('object_type', 'object_id')
```
model_to_dict
```
result = []
                for obj in queryset:
                    print obj.tags
                    result.append(model_to_dict(
                        obj, recurse=True, backrefs=True))
```
note: model create dynamically using : 
```
    def generate_XXXX_model(self):
        model_fields = {
            'text': CharField(index=True),
            'tags': ReverseGFK(Tag, 'object_type', 'object_id')
        }
        model = type(self.model_name, (BaseModel,), model_fields)
        model._meta.db_table = "XXXX_%s_XXXX" % self.table_name
        if not model.table_exists():
            model.create_table()
        _XXXX_XXX_models_cache[self.table_name] = model
        return model
```
result sample:
```
[
    {
        "username": "user3",
        "created": "2017-08-16T04:12:24.374450",
        "text": "api",
        "modified": "2017-08-16T04:12:24.374481",
        "feature_id": "11",
        "id": 2
    },
    {
        "username": "user2",
        "created": "2017-08-16T04:14:17.899443",
        "text": "api",
        "modified": "2017-08-16T04:14:17.899524",
        "feature_id": "13",
        "id": 3
    },
    {
        "username": "user1",
        "created": "2017-08-16T04:14:50.413285",
        "text": "api",
        "modified": "2017-08-16T04:14:50.413322",
        "feature_id": "16",
        "id": 4
    }
]
```

The `model_to_dict` accepts a parameter `extra_attrs` which you can use to do the lookup on the GFK object.
Issue: insert many and ids
Author: ak4nv
State: closed
Created: 2017-08-12 07:57:42

Hi!
Is it possible to insert many records and get them id list?
Or maybe id of the last inserted. Or get the next identity value?
In one transaction. In postgresql.
Would be awesome!

Currently if you are using Postgresql you can write:

```python
query = User.insert_many([{...}, {...}, {...}]).return_id_list()
ids_inserted = query.execute()
```

It's admittedly a weird API and not very flexible. In 3.0a it is the default to return the id list when the database supports the returning clause.
Issue: Potential mistake in docs regarding operator parameter in CompoundSelect
Author: aniketpanjwani
State: closed
Created: 2017-08-11 23:38:07

I'm using peewee 2.9.1 with Python 3.5.2 on Linux Mint 18 to access a PostgreSQL 9.6.2 database hosted on AWS RDS.

The [documentation for CompoundSelect](http://docs.peewee-orm.com/en/latest/peewee/api.html#CompoundSelect) states that one should use a Node instance for the operator parameter in a compound query. I tried each of SQL('UNION'), SQL('INTERSECTION'), and SQL('EXCEPT'), but kept getting errors from psycopg2 and peewee with the syntax of the formatted SQL.

An example of the formatted SQL is below:

```
>>> print(inds.sql()
```
`('(SELECT "t1"."id", "t1"."recid", "t1"."year", "t1"."papername", "t1"."timeofday", "t1"."newspaper_id" FROM "newspaperyears" AS t1 WHERE ("t1"."year" = %s)) <peewee.SQL object at 0x7f2d702d6518> (SELECT "t3"."id", "t3"."recid", "t3"."year", "t3"."papername", "t3"."timeofday", "t3"."newspaper_id" FROM "newspaperyears" AS t3 INNER JOIN "newspapers" AS t4 ON ("t3"."newspaper_id" = "t4"."id") WHERE ("t3"."year" = %s))', [1952, 1952])`

It seems like either peewee's or psycopg2's SQL generation was directly inserting a reference to the Node instance. Noticing this, I instead simply used 'EXCEPT' instead of SQL('EXCEPT') for the operator parameter, and I was able to get the CompoundSelect query that I wanted.

I checked the commit history, and it didn't seem like there has been anything addressing this type of issue between 2.9.1 and 2.10, but my apologies if I missed something.

Wow, no the docs are definitely wrong. I'll update them, thanks.
Issue: execute_sql, unexpected result
Author: pepelisu
State: closed
Created: 2017-08-09 13:49:26

Hi, 
While I was trying to perform a cleaning operation in a database, specifically removing the duplicates in a SQLite database.
Using a piece of code similar to this:
```python
db.execute_sql("DELETE FROM tableName WHERE rowid NOT IN (SELECT MIN(rowid) FROM tableName GROUP BY 'column1','column2','column3')")
```
The result is a complete delete of all entries in the database but the first one. It seems that execute_sql is not performing the grouping. 
If I run exactly the same query in the database with sqlite via terminal the result is the expected one. All the duplicated records are removed. 
I am doing something wrong or is there a bug?



There must be a problem with the query as written. Peewee doesn't do anything to SQL queries passed directly to execute_sql() -- they are passed to the driver, which then executes them. I'd suggest working on the query in the database shell until you get the desired results.

You are right, I will keep working in the query, since it seems to work with DB browser, but not in sqlite3 in terminal. 
The problem was simple quotation marks, it has to be double to escape names.
Issue: fix postgresql use DictCursor save bug
Author: mkxxq
State: closed
Created: 2017-08-02 15:31:30

Hi:
When peewee in postgresql and use DictCursor that one model with a not char pk use 'save' is error.
test_code is :
```
from psycopg2.extras import RealDictCursor
from peewee import *
test_psql_db = PostgresqlDatabase('peewee_test', host='127.0.0.1', port=5432,
                                  user='postgres', password='postgres',

                                  cursor_factory=RealDictCursor)
class IntPKModel(Model):
    pk = PrimaryKeyField()
    data = CharField()

    class Meta:
        database = test_psql_db
class TestIntPKModelWithDictCursor(ModelTestCase):
    requires = [IntPKModel]

    def test_save(self):
        pi = IntPKModel(data='pi1')
        pi.save()
```

error like that:
```
======================================================================
ERROR: test_save (playhouse.tests.test_models.TestIntPKModelWithDictCursor)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/xxq/prv_workspace/peewee/playhouse/tests/test_models.py", line 2368, in test_save
    pi.save()
  File "/home/xxq/prv_workspace/peewee/peewee.py", line 5385, in save
    pk_from_cursor = self.insert(**field_dict).execute()
  File "/home/xxq/prv_workspace/peewee/peewee.py", line 3746, in execute
    pk_row)]
  File "/home/xxq/prv_workspace/peewee/peewee.py", line 3744, in <listcomp>
    for field, column
  File "/home/xxq/prv_workspace/peewee/peewee.py", line 1096, in python_value
    return value if value is None else self.coerce(value)
ValueError: invalid literal for int() with base 10: 'pk'
```

wtf is this garbage? I can't tell what you changed