akumoth/peewee-issues huggingface dataset:
Issue: Default Snake Case Table Names
Author: joshuamcginnis
State: closed
Created: 2017-03-22 23:32:15

I have several tables that have camelcased names (`FooType`, `FooScoreAttributes`, etc) and on table creation, the table names are: `footype` and `fooscoreattributes`.

Is there a way to have all table names snake cased  like: `foo_type`  without having to specify the `db_table` in each model?

Sorry for the inconvenience, but you'll need to specify the `db_table` attribute.

Alternatively, you can pass in a function using `db_table_func` that does the conversion the way you'd prefer. Something like:

```python
def make_table_name(model_cls):
    name = model_cls.__name__
    # change name to snake-case
    return new_name

class BaseModel(Model):
    class Meta:
        db_table_func = make_table_name

class FooType(BaseModel):
    ...

class FooScoreAttributes(BaseModel):
    ...
```

@coleifer Thanks! `db_table_func` is just what I need.
Issue: PostgreSQL - range types
Author: Phibedy
State: closed
Created: 2017-03-22 11:56:22

Hi,
would like to use [range types](https://www.postgresql.org/docs/9.3/static/rangetypes.html) (in my case datetime with time zone), but I am not sure how to implement it for all types.
Any help appreciated :)

I'm going to pass on implementing these right now, though I may revisit this feature in the future. If you're looking for ideas on implementation, I'd suggest checking out the other custom field types in `playhouse.postgres_ext`.
Issue: Inconsistent behaviour from commit() in nested db.atomic() (PostgreSQL)
Author: Syeberman
State: closed
Created: 2017-03-21 21:18:09

`db.atomic()` returns either a transaction or a savepoint, depending on the nesting level.  Manually committing or rolling back a transaction starts a new transaction, but *not* for savepoints.  In PostgreSQL, it is valid to [roll back](https://www.postgresql.org/docs/current/static/sql-rollback-to.html) a savepoint multiple times, but you can only [commit once](https://www.postgresql.org/docs/current/static/sql-release-savepoint.html).

The net result of all this is inconsistent behaviour from the object returned by `db.atomic()`:

```python
class Test(Model):
    value = CharField()

    class Meta:
        database = db

db.connect()
db.drop_table(Test)
db.create_table(Test)

with db.atomic() as trans:
    Test.create(value='trans rollback')
    trans.rollback()

    Test.create(value='trans commit')
    trans.commit()  # success

    with db.atomic() as savep:
        Test.create(value='savep rollback')
        savep.rollback()

        Test.create(value='savep commit')
        # savep.commit()  # `InternalError: no such savepoint` on __exit__

for test in Test.select():
    print(test.value)
```

I believe that the object returned from `db.atomic()` should *always* behave like a transaction, where manually committing or rolling back starts a new transaction.  The code above makes it obvious that there are nested transactions, but consider a helper method that sometimes is called in a transaction and sometimes not: that method cannot manually commit.

Thanks, fixed.
Issue: pip3 install peewee error m wrong cython version
Author: stanta
State: closed
Created: 2017-03-18 05:18:01

Hi, when I install peewee from pip, have an issue: ValueError: invalid version number '0.20.1post0'
on vagrant/   Ubuntu Trusty 14.04.5 LTS, says: 
    **elif StrictVersion(cython_version) < StrictVersion(cython_min_version):**

look:
root@lists:~/.pip# sudo pip3 install peewee
Downloading/unpacking peewee
  Downloading peewee-2.9.1.tar.gz (514kB): 514kB downloaded
  Running setup.py (path:/tmp/pip_build_root/peewee/setup.py) egg_info for package peewee
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/pip_build_root/peewee/setup.py", line 29, in <module>
        elif StrictVersion(cython_version) < StrictVersion(cython_min_version):
      File "/usr/lib/python3.4/distutils/version.py", line 40, in __init__
        self.parse(vstring)
      File "/usr/lib/python3.4/distutils/version.py", line 143, in parse
        raise ValueError("invalid version number '%s'" % vstring)
    ValueError: invalid version number '0.20.1post0'
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/tmp/pip_build_root/peewee/setup.py", line 29, in <module>

    **elif StrictVersion(cython_version) < StrictVersion(cython_min_version):**

  File "/usr/lib/python3.4/distutils/version.py", line 40, in __init__

    self.parse(vstring)

  File "/usr/lib/python3.4/distutils/version.py", line 143, in parse

    raise ValueError("invalid version number '%s'" % vstring)

ValueError: invalid version number '0.20.1post0'

----------------------------------------
Cleaning up...



That does indeed look like an invalid version number. Plus it's too old at any rate.

yeah, I downloaded cython sources from repo and go through "python setup install" - all became OK
Issue: Peewee writing to peewee.db not my.db
Author: mtairu
State: closed
Created: 2017-03-17 18:43:28

When I insert items to db, it writes to peewee.db not my.db

Check that your model is specifying the database as a meta option:

```python
my_db = SqliteDatabase('my.db')

class MyModel(Model):
    class Meta:
        database = my_db
```

For what it's worth (I can't believe I'm even taking the time to type this), don't you imagine if something as fundamental as writing to the "wrong" database were happening, that this would be a massive bug? That the tests would probably be failing? Etc? Did it not occur to you that you had made an error?

sorry coleifer.. you correction worked and has been working. Just forgot to comeback and say so.

To think, I wanted to learn sqlalchemy.... not worth it. Peewee is good enough. Just having difficulty sorting my datetime values 2017-04-07T01:51:00.000Z all the way to time. It sorts the dates but the time is always an issue I tried everything.

Are you using 24-hour encoding? They should sort correctly simply due to being in order alphabetically. Of course if time-zones are introduced that may not be the case.
Issue: Use `get_primary_key_fields()` instead of `primary_key`, because the former knows how to handle composite keys
Author: Necoro
State: closed
Created: 2017-03-12 09:14:08

There is already a method `get_primary_key_fields()` which does just what is expected: return the list of all primary key fields, also handling CompositeKeys just right.

There are probably some more instances throughout the code, where using that method instead of `primary_key` would be advisable.

This should fix isse #1219.

Nice, thanks!

I also merged another fix that should also take care of any other instances of the issue.
Issue: CompositeKey in subquery not resolved
Author: Necoro
State: closed
Created: 2017-03-12 08:36:14

Given the following simple model with a composite PK:
```python
class DocumentTag(BaseModel):
    document = ForeignKeyField(Document, related_name = 'tags')
    tag = ForeignKeyField(Tag)

    class Meta:
        primary_key = CompositeKey('document', 'tag')
```

`DocumentTag.select()`  works as expected, yielding
```sql
SELECT "t1"."document_id", "t1"."tag_id" FROM "documenttag" AS t1
```

When I now want to use `DocumentTag` in a subquery, the CompositeKey is not resolved as one would expect (the query does not make sense, but works for illustration purposes):
`Document.select().where(DocumentTag.select() == 1)` now yields
```sql
SELECT "t1".*
FROM "document" AS t1 
WHERE ((SELECT ? FROM "documenttag" AS t2) = ?) 
[<peewee.CompositeKey object at 0x7f1c0465b438>, 1]
```

The same holds for other uses of subqueries, like joining with a subquery.
Issue: Test failure in 2.9.0 release
Author: felixonmars
State: closed
Created: 2017-03-10 04:56:32

```
======================================================================
ERROR: test_create_or_get (playhouse.tests.test_gfk.GFKTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/python-peewee/src/peewee/playhouse/tests/test_gfk.py", line 123, in test_create_or_get
    self._test_get_create(Tag.create_or_get)
AttributeError: type object 'Tag' has no attribute 'create_or_get'

----------------------------------------------------------------------
Ran 598 tests in 5.246s

FAILED (errors=1)
```

Yeah, it's fixed.
Issue: Add most recent Python versions in Travis CI
Author: SylvainDe
State: closed
Created: 2017-03-09 19:14:05

Add more recent Python versions including development branches and nightly build.

The motivation came from reading brettcannon's article : https://snarky.ca/how-to-use-your-project-travis-to-help-test-python-itself/ . Trying to activate the newest Python versions on CI jobs is in most case a win-win situation: if everything works fine, there is nothing to worry about. If an issue is spotted, it is good to know about it to fix it on your side or to open a bug on CPython ( https://bugs.python.org/ ).

Also, if a failures is spotted, you can use the allow_failures option in your matrix build (more information about this in the link above).

Also, nightly and 3.7-dev may be a bit too recent/unstable so I can remove then if need be.

More information about how this PR happened : https://github.com/SylvainDe/CIthon .

More information about the various Python versions available on Travis : https://docs.travis-ci.com/user/languages/python/ .

More information about the Python 3.6 version schedule : https://www.python.org/dev/peps/pep-0494/ .
More information about the Python 3.7 version schedule : https://www.python.org/dev/peps/pep-0537/ .
Issue: Hack: Peewee Tables to Pandas DataFrames
Author: NJDFan
State: closed
Created: 2017-03-09 00:37:27

I find SQL tables excellent for experimental data collection (often multi-day processes) and godawful for examination/analysis.  So, I figured out this trick and figured I'd send it to you in case anyone else out there finds it useful.  Note: it would definitely be cleaner if there was a mechanism to introspect fields.

```python
import pandas as pd
import peewee

def peewee_to_df(table):
    fields = [fld for fld in dir(table) if isinstance(getattr(table, fld), peewee.Field)]
    data = list(table.select(*[getattr(table, fld) for fld in fields]).tuples())
    df = pd.DataFrame.from_records(data, columns=fields)
    if 'id' in fields:
        df.set_index('id', inplace=True)
    return df
```



There is a mechanism to inspect fields, MyModelClass._meta.fields. Check it out -- should be what you need.
Issue: Update cache without table name throws error
Author: erichaydel
State: closed
Created: 2017-03-08 22:30:01

In line 82 of dataset, https://github.com/coleifer/peewee/blob/master/playhouse/dataset.py#L82, table_names is calculated as a list inline.
However, if no table name is set (want to update all), dependencies is set to None a couple of lines earlier.
This will throw the error "NoneType is not iterable".
Issue: select without commit?
Author: knightliao
State: closed
Created: 2017-03-06 03:57:20

i found that every select sql will do commit at last in peewee.
this may cause transaction.

mybatis is a famous lib, it doesn't do transaction when it finds that select sql. 

will peewee implement " select without commit?" function? 

Peewee runs in autocommit mode unless you're within the scope of a transaction block, so any COMMIT issued as part of a SELECT query is occurring because a transaction is *not* active. If you're managing transactions outside of Peewee, then you may want to turn autocommit off (by passing `autocommit=False` in to the database constructor). I'd recommend against doing that, however.

Assuming you're running transactions using either the `atomic()` or `transaction()` helpers, then there's nothing to worry about.

thanks a lot :)
Issue: Add interval field support for Postgres
Author: alexlatchford
State: closed
Created: 2017-03-06 00:06:03

I can add/amend the tests if you think this is a good approach.

PS. Sorry looks like my AutoPEP8 got in the way of the changes, I can remove if needs be.

What Python data-type is used by psycopg2 to represent an interval in Postgres? I'd assume a `datetime.timedelta`? I'm good with merging the changes (though please rebase into a single commit and remove the PEP8 changes).

Yep uses `timedelta` internally. Should be all handled transparently by `psycopg2` and the operations by the operators on `Node` from my initial tests.

Rebased as suggested and removed formatting.

Looks like the test model differs from the model used in the test-case itself, which is causing the builds to fail. You mind fixing that, and maybe adding an integration test that actually creates the table and verifies the data and data-types being stored/retrieved from the DB?

It was so close! Just needed to fix the tests up a bit. Any reason you closed this?

Unfortunately the project I needed this for was cancelled and I'm unlikely to get around to fixing this up anytime soon so didn't want to clutter things up. Feel free to take it on if you think it's a good addition :)

Thanks @alexlatchford -- I may do that when I've got a little free time. I appreciate the effort!

Committed f4ebd78b990efc6f0fc57b9a4d81f0cd6ecdf276
Issue: Make 'create_or_get' correctly support composite unique indexes and keys
Author: Necoro
State: closed
Created: 2017-03-05 20:32:09

This fixes #1204 

Hmm... I received another pull-request with similar changes a while back and passed on it due to the complexity it introduced. My current thinking is that it'd be better to just remove `create_or_get` since it seems to cause confusion and the code one would need to write to work around it is very concise:

```python
try:
    return MyObj.create(whatever='foo')
except IntegrityError:
    return MyObj.get(MyObj.whatever == 'foo')
```

To me the result does not justify the means (and the subtlety involved). So I think in the next minor release I'll just remove the method instead.
Issue: create_or_get returns wrong instance
Author: Necoro
State: closed
Created: 2017-03-05 19:23:55

```python
>>> Tag.get(name='etf', prefix='wp')
<Tag (#16) wp:etf>
>>> Tag.create_or_get(name='etf', prefix='wp')
(<Tag (#1) mime:*>, False)
>>> Tag.get_or_create(name='etf', prefix='wp')
(<Tag (#16) wp:etf>, False)
```

See above, should be self explanatory. Going to analyze (and hopefully fix) it now.

Is the create not raising an IntegrityError in the create_or_get case?

The problem was a combined unique index. My PR should fix it.

Thanks for your work on the PR. As I said in my comment on the PR, the code one needs to write to do a create-or-get is only a few lines long and, in my opinion, does not justify the complexity of the corresponding helper method. In the next minor release, I'll be removing `create_or_get()` since it seems to cause more confusion that actually help.

That is a very good idea. I wasn't sure if I could suggest this.

In both the old and my improved state, there exist configurations (e.g. multiple unique indexes) where the wrong entry would be returned. Heck, it might not even be possible to say what the right entry would be: If you tried to insert an entry violating _two_ unique indexes at the same time, the method could return the entry for either of those.

Exactly. For only the most trivial use-cases would that method be very helpful, and the resulting code to achieve the equivalent functionality is only a couple extra lines... Thanks for your work on this PR, I hope you don't feel it was time wasted.
Issue: SQLite Closure Tables: Allow for many-to-many relations
Author: Necoro
State: closed
Created: 2017-03-04 20:39:55

Currently, the ClosureTable only allows for self-joins using the same table. But if you have many-to-many relations between one table using an intermediate table that was not expressible.

With this PR, a self-many-to-many-relation is now supported (as rare as it might be -- I needed it ;)).

I also took the liberty to update the documentation, but this probably could use a second look. 

Cool, I may make some small tweaks here and there, but looks great, thank you!

Actually, I've gone ahead and reverted this. It didn't seem to work when I started writing tests for it -- possibly I'm misunderstanding something?

Could you give me an example of what was not working as expected?

Ach, I take it back! I think I must've introduced a bug when I was editing the code to clean up some of the names.

Changes merged, I swapped the `id_column` parameter for one named `referencing_key` instead and made them `VirtualIntegerFields`. Other than that, changes are intact.

https://github.com/coleifer/peewee/blob/master/playhouse/sqlite_ext.py#L670-L758

Thanks. The fields `idcolumn` and `parentcolumn` are still strings, though. See the docs from closure.c:
```sql
SELECT element.* FROM element, ct1
WHERE element.groupid=ct1.id
AND ct1.root=?1
AND ct1.idcolumn='parentId'
AND ct1.parentcolumn='groupId';
```

Oh shit. Thanks! Will re-fix your fix.

04a3e022144ebf3db6a078ecd83f51dbbf766034
Issue: Make Model.get(val = None) work the same as Model.select().where(Model.val == None)
Author: Necoro
State: closed
Created: 2017-03-04 19:51:37

Currently `Model.select().where(val == None)` compiles into a `WHERE val IS NULL` clause. On the other hand `Model.get(val = None)` compiled into `WHERE val = NULL` and, if val is an FK field, tried to join.

With this PR, this is fixed and Model.get behaves just like Model.select().where()

Nice, thanks!
Issue: After updating to 2.8.8 from 2.8.5, started seeing a bug that occurs on 2.8.7 as well
Author: savraj
State: closed
Created: 2017-02-27 23:28:06

Hi Charles and team,

First, many thanks for creating an AMAZING ORM -- we depend on it and we love its simplicity.

So the weird bug is, we see a
`OperationalError('Connection already open')`

In our boilerplate Flask connection open/close code (copied from the docs), and it happens only on the first time our server comes up running on the google app engine development server.

```
# set up DB handling
@app.before_request
def connect_db():
    our_db.connect()  # this raises an operational error only on the first request

@app.teardown_request
def close_db(exc):
    if not our_db.is_closed():
        our_db.close()
```

Thing is, I can move back to 2.8.5 and everything works normally -- no OperationalError is raised.  Though we looked at the commits between .5 and .8 and didn't see anything unusual. Your insight is appreciated! Thank you in advance, and again, we love Peewee!!! (Our temporary fix is sticking with 2.8.5)

-savraj


I bumped into this too. The commit that causes this is 9e76c999868197a4911fa2fa457c50ab42431ff1, which came about because of #1108.

The pull request explains why the commit is necessary and how you can solve your problem, but in short use `our_db.get_conn()` instead of `our_db.connect()`.

If this is a permanent solution then the docs should probably be updated (especially the [Adding Request Hooks](http://docs.peewee-orm.com/en/latest/peewee/database.html#adding-request-hooks) section), but I'll leave that up to @coleifer to decide.

Thanks for the quick reply Matt, much appreciated!

Also you can use [`FlaskDB`](https://github.com/coleifer/peewee/blob/master/playhouse/flask_utils.py#L78) from `playhouse.flask_utils` module

Related: Where are the release notes for 2.8.5 through 2.8.8? :)

> In our boilerplate Flask connection open/close code (copied from the docs), and it happens only on the first time our server comes up running on the google app engine development server.

Are you doing anything with the database at module-scope? For instance, creating tables, or anything that might unintentionally cause a connection to be opened?

@flother -- thanks, by the way, for providing the details on the commit and a workaround.

The change prevents connections from being opened and then subsequently not closed because the reference to them was overwritten with a 2nd connection.

The OperationalError is clue-ing you in to a spot in your code where you've already got an open connection for the given thread and are trying to open a new one on top of it. Like I said -- check for database accesses at module scope or during application startup, those are likely to be the culprits.

As for using `get_conn()` in the hooks, I still think it is advisable to use `connect()`, because in most applications an exception would only be raised because the developer had forgotten to close one somewhere...which seems like a bug.
Issue: ForeignKey: Do not load object if only the pk is accessed
Author: Necoro
State: closed
Created: 2017-02-26 20:11:51

Given two models `Tweet` and `User` with the well-known relations: When accessing `tweet.user` Peewee currently always fetches the complete user.

Even in cases like
```python
if tweet.user:
    # we have a user, do something
else:
    # no user, eh?
```
and
```python
if tweet.user == someUser:
    # do something
```

In my opinion, in both cases the loading is not needed because it is sufficient to check whether `tweet.user_id is not None` and `tweet.user_id == someUser.id`, respectively. (And btw, that you can access the ids directly that way is not documented, is it? I had to inspect peewee's code to find it).

I think, that implementing this behavior should be fairly straight-forward. If you are not opposed, I'd give it a try.

(Of course, this only works with enabled foreign keys, so might be a trap for incautious sqlite users.)

I think that we shouldn't break the convention that accessing the foreign key will, if necessary, load the related object. It makes the code simpler, too.

Just compare the IDs if that's all you need:

```python
if tweet.user_id:
```

Or

```python
if tweet.user_id == someUser.id:
```
Issue: db.atomic() does not set transaction_type
Author: jberkel
State: closed
Created: 2017-02-25 17:01:42

Regression after updating from 2.8.5 to 2.8.7:

```
  File "/Users/jan/projects/lexi/importer/src/main/python/lexi/models/fts.py", line 160, in index
    with database.atomic():
  File "/Users/jan/projects/lexi/importer/venv-python2/lib/python2.7/site-packages/peewee.py", line 4356, in __enter__
    return self.context_manager.__enter__()
  File "/Users/jan/projects/lexi/importer/venv-python2/lib/python2.7/site-packages/peewee.py", line 4381, in __enter__
    if self.db.transaction_depth() == 0: self._begin()
  File "/Users/jan/projects/lexi/importer/venv-python2/lib/python2.7/site-packages/playhouse/apsw_ext.py", line 46, in _begin
    self.db.begin(self.lock_type)
  File "/Users/jan/projects/lexi/importer/venv-python2/lib/python2.7/site-packages/playhouse/apsw_ext.py", line 127, in begin
    self.get_cursor().execute('begin %s;' % lock_type)
  File "src/cursor.c", line 1019, in APSWCursor_execute.sqlite3_prepare
  File "src/statementcache.c", line 386, in sqlite3_prepare
SQLError: SQLError: near "None": syntax error
```

class Database:
```python
def atomic(self, transaction_type=None):
        return _atomic(self, transaction_type)
```

```python
class _atomic(_callable_context_manager):
   def __enter__(self):
        if self.db.transaction_depth() == 0:
            self.context_manager = self.db.transaction(self.transaction_type)
   ...
```

class APSWDatabase:
```python
def begin(self, lock_type='deferred'):
        self.get_cursor().execute('begin %s;' % lock_type)
                                                ^^^^^^^^^ = None
```


Thanks for the report, pushed 2.8.8.