akumoth/peewee-issues huggingface dataset:
Issue: Aggregate(...) and fn.Max for DateTimeField should return datetime objects
Author: passiomatic
State: closed
Created: 2013-06-26 20:23:13

I'm using Peewee 2.1.2, Python 2.6.1 with bundled sqlite3, see versions below: 

```
>>> sqlite3.sqlite_version_info
(3, 6, 12)
>>> sqlite3.version_info
(2, 4, 1)
```

Test case to reproduce the problem:

``` python

from datetime import datetime
from peewee import *

the_db = SqliteDatabase(':memory:') 

class MyModel(Model):
    dt = DateTimeField(null=True)

    class Meta:
        database = the_db

if __name__ == '__main__':

    the_db.connect()
    MyModel.create_table()

    # Fill    
    now = datetime.now()    
    for _ in range(50):
        m = MyModel()
        m.dt = now    
        m.save()

    # This works
    q = MyModel.select()            
    for i in q:
        assert type(i.dt) is datetime

    # These both fails    
    r = MyModel.select().aggregate(fn.Max(MyModel.dt))
    print type(r)
    assert type(r) is datetime

    q = MyModel.select(fn.Max(MyModel.dt).alias('max_dt'))    
    for i in q:
        print type(i.max_dt)
        assert type(i.max_dt) is datetime

```

Output:

```
Underworld:$ python testcase.py 
<type 'unicode'>
Traceback (most recent call last):
  File "testcase.py", line 32, in <module>
    assert type(r) is datetime
AssertionError
```

Thank you.


Unfortunately, I'm not sure there's a great solution for this without resorting to hacks.  It would be possible to special-case flat aggregation / annotations by introspecting the field type and then passing the value from the cursor through the field's `python_value` method, but I'm not sure how clean the implementation would be and it might be fragile.  The other thing I want to avoid is a performance penalty.

For now I am going to leave this open, but I'm not sure that I will be able to resolve it.

Note:  This problem only appears in Sqlite and not mysql / postgres.


Partially fixed in d23619abeb0f78d46284658d012970a30c7e3186 .  `annotate` works now, just `aggregate` / `scalar` fail.


That's great, thank you. Do you plan to do a dot release soon?


Within the next day or two.
Issue: Selecting multiple models / naive query
Author: coleifer
State: closed
Created: 2013-06-17 03:17:46

It appears peewee is not correctly assigning attributes from a related model:

``` python
from peewee import SqliteDatabase, Model, PrimaryKeyField, CharField, \
  IntegerField, ForeignKeyField
from playhouse.test_utils import test_database

test_db = SqliteDatabase(':memory:')

class Model1(Model):
  id = PrimaryKeyField()
  another_id = IntegerField()
  field = CharField()

  def __str__(self):
    return self.__class__.__name__ + '[' + ', '.join(["%s: %s" % (k, v) for k, v in self._data.items()]) + "]"

class Model2(Model):
  some_id = IntegerField() # PK
  another_id = IntegerField() # PK
  field2 = CharField()

  def __str__(self):
    return self.__class__.__name__ + '[' + ', '.join(["%s: %s" % (k, v) for k, v in self._data.items()]) + "]"

  class Meta():
    indexes = ((('some_id', 'another_id'), True),)

if __name__ == '__main__':

  with test_database(test_db, (Model1, Model2)):
    Model1.\
     create(id=1, another_id=1, field="example").\
     create(id=2, another_id=3, field="example")

    Model2.\
     create(some_id=1, another_id=1, field2="example2").\
     create(some_id=1, another_id=2, field2="example2").\
     create(some_id=2, another_id=3, field2="example2")

    res = (Model1
           .select(Model1, Model2)
           .join(Model2, on=(
               (Model1.id == Model2.some_id) &
               (Model1.another_id == Model2.another_id)
           ))
           .naive())
    print res.sql

    for a in res:
      print a
```
Issue: Choices attribute according to documentation
Author: niedbalski
State: closed
Created: 2013-06-16 15:10:06

Hello!,

Added a validation for the choices field attribute. I started looking at this issue because WTForms-peewee raised an 'unpack error' using a field declared as Charfield(choices=('OPTION', 'OPTION')) , according to the peewee documentation this must be specified as (value, label) so i added a ValueError exception is the iterable don't follow this specification.

Also i added a test for this field attribute.

Regards


I think I will pass on merging this as it consumes the choices iterable once upon field initialization.  Rather, I will rely on the documentation.


OK, perfect.I always prefer the code that dies descriptively
Issue: Add sample encoding setting
Author: rcarmo
State: closed
Created: 2013-06-11 16:02:25

This will save non-English speakers hours of puttering around.


Thanks @rcarmo!


Don't mention it. Tell you what, I'll see if I can add more surgical samples like this - I'm using Peewee for a lot of stuff lately, and have all sorts of little recipes... 

On Jun 11, 2013, at 17:04 , Charles Leifer notifications@github.com wrote:

> Thanks @rcarmo!
> 
> —
> Reply to this email directly or view it on GitHub.
Issue: MySQL support for Python 3
Author: davidak
State: closed
Created: 2013-06-03 05:19:34

the last release of MySQLdb is from 02.11.2012 (MySQLdb-1.2.4 release candidate 1) and it doesn't support python 3.

maybe you or someone else have the time to let peewee support another MySQL Driver?

This one supports Python 3 already and is developed by Oracle:
https://pypi.python.org/pypi/mysql-connector-python

its very bad for me. i coded the whole night and now if i want to deploy my great app i get lots of errors.
you say that peewee supports python3 and mysql. too bad not at the same time.
so i will switch to sqlalchemy with mysql-connector driver.

or is there a quick solution to get it work for me?

```
  File "./import.py", line 34, in erstelle_tabellen
    mysql.connect()
  File "/home/davidak/websites/satzgenerator/lib/python3.2/site-packages/peewee.py", line 1756, in connect
    self.__local.conn = self._connect(self.database, **self.connect_kwargs)
  File "/home/davidak/websites/satzgenerator/lib/python3.2/site-packages/peewee.py", line 2011, in _connect
    return mysql.connect(db=database, **conn_kwargs)
  File "/home/davidak/websites/satzgenerator/lib/python3.2/site-packages/MySQLdb/__init__.py", line 80, in Connect
    from MySQLdb.connections import Connection
  File "/home/davidak/websites/satzgenerator/lib/python3.2/site-packages/MySQLdb/connections.py", line 36
    raise errorclass, errorvalue
                    ^
SyntaxError: invalid syntax
```


Sorry to hear you had trouble with mysqldb.  I'm not familiar with the Oracle driver, but there is a python-3 compatible pure python driver which will work automatically with peewee:

https://github.com/petehunt/PyMySQL

The MySQLdb project does not yet support python 3, and in fact peewee just got py3k support a month or two ago.  I will add a note to the mysql section of the docs indicating this.  To fix your error, uninstall the mysqldb package and try pymysql.  Good luck!


Ok, i tested it with pymysql, but get also an error.
https://github.com/petehunt/PyMySQL/issues/151

i have reported it to the developer, but don't get a reaction.
in his blog he announced, that he will not maintain PyMySQL further.
http://www.petehunt.net/blog/?p=49

i tested SQLAlchemy with the MySQL Connector and it works.
their SQL Expression Language is nice to write and i don't see the advantage of an orm in my small project.

so im using that now and next time i test it on my server before programming 2 nights and it don't work.
maybe you can support MySQL Connector if you have the time, but i don't need it anymore.


PyMySQL is under active development again and it supports Python 3:

https://github.com/PyMySQL/PyMySQL

I'm using it along with peewee and had no problems so far.


@lecram great to hear and thanks for fixing that bug.
Issue: Datetime fields with timezone
Author: rcarmo
State: closed
Created: 2013-06-01 17:31:44

This might be just me, but I cannot seem to be able to create DateTime fields with a timezone under Postgres. The [docs](http://www.postgresql.org/docs/9.1/static/datatype-datetime.html) state tacking on "with time zone" to the field spec should do it, but I haven't been able to figure out how to specify that in a Peewee model successfully.

Could you please check if that is possible and, if so, add an example for that to ReadTheDocs?

Thanks!


It is fairly straightforward to add support for custom fields: http://peewee.readthedocs.org/en/latest/peewee/models.html#creating-a-custom-field

I also added a `DateTimeTZField` to the `playhouse.postgres_ext` module.


Awesome, thanks!
Issue: Field type CharField limited to 255 characters.
Author: dstruck
State: closed
Created: 2013-05-31 08:42:56

The field type "CharField" gets truncated to 255 characters.  You can define VARCHAR fields in PostgreSQL of size bigger then 255.

Maybe one could raise an exception if a value gets cut off?

Test case:

``` python
'''
-- create test table in postgresql

CREATE TABLE sample(
id          SERIAL PRIMARY KEY
,data       VARCHAR(3000)
);
'''

################################################################################
# classes generated by pwiz.py
# pwiz.py -e postgresql test > model.txt
from peewee import *

database = PostgresqlDatabase('test', **{})

class UnknownFieldType(object):
    pass

class BaseModel(Model):
    class Meta:
        database = database

class Sample(BaseModel):
    data = CharField(null=True)

    class Meta:
        db_table = 'sample'




################################################################################

# test
sequence = "Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum."

print "string length before submission:",len(sequence)

# save data in database
entry = Sample.create(data=sequence)

# retrieve data back from database
entry_db = Sample.get(Sample.id == entry.id)
print "string length in database after submission:",len(entry_db.data)

'''
Output:
string length before submission: 574
string length in database after submission: 255    
'''
```


Unfortunately, Pwiz does not introspect the size the a VARCHAR field.

Additionally, in order to not break compatibility with older versions of peewee, the `max_length` attribute defaults to `255`.  So the following are equivalent:

``` python

foo = CharField()
bar = CharField(max_length=255)
```

If you would like to use 3000 characters:

``` python
baz = CharField(max_length=3000)
```

Or even better,

``` python
baz = TextField()
```


Yes, that's what I did finally (CharField to TextField).

It might be an idea to avert the user that the values have been cut off. Maybe raise an exception?


> Maybe raise an exception?

That seems like a good plan.  Thanks, and sorry for the confusion.
Issue: Restore SQLite Full Text Search Support
Author: wearpants
State: closed
Created: 2013-05-30 23:58:51

There [used to be](http://charlesleifer.com/blog/using-advanced-database-features-with-peewee-a-python-orm/) support for FTS3/4 in SQlite, but it went away some time.

Bring it back, for great justice!


It's back!  0519ac68246a3a73aca9e653c1841183642061fd

I mostly upgraded this from the old code, and still have some cleanups planned so be warned, the API may change a bit!
Issue: add connections pool support
Author: beanyoung
State: closed
Created: 2013-05-25 09:31:02

The connections pool is based on DBUtils and it works fine for me except that count() method of SelectQuery always return 0. But I can get the count with

``` python
Foo.select(fn.Count(Foo.id).alias('count')).get().count
```


Interesting, I had not heard of dbutils.  Thanks for sharing this!

Regarding merging this, a third-party library dependency is not something I want to add to peewee.  It might work better as a module in `playhouse`, though -- this is where I've been buliding little extensions to peewee.  Another note, if you're wanting to merge in new functionality for peewee, I also like there to be tests and documentation.

Thanks again!
Issue: Model.create() doesn't set id field for BigIntegerField primary key
Author: rudyryk
State: closed
Created: 2013-05-22 18:56:54

Hi! Peewee is awesome lib! :)

It sees there's an issue with BigIntegerField as primary key.

I define my model like that:

```
import peewee
db = peewee.SqliteDatabase('test.db')
class User(peewee.Model):
    id = peewee.BigIntegerField(primary_key=True)
    token = peewee.CharField(max_length=100, default='')
    class Meta:
        database = db
```

Then I test create method:

```
>>> User.create_table()
>>> user = User.create(token="123")
>>> user.id
>>> 
>>> user = User.get(token="123")
>>> user.id
1
```

Instantiating `User(token="123")` and then `save()` results the same.

If I remove line `id = peewee.BigIntegerField(primary_key=True)` everything works fine. I've used Python 3.3.1, may be it's only py3 issue, not sure about that.


Thanks glad you like the library!

Unfortunately, peewee's `PrimaryKeyField` is the only field that will handle auto-incrementing primary keys.  If you want to use the BigIntegerField as an auto-incrementing primary key you will need to create a custom field type (probably by subclassing `PrimaryKeyField`):

http://peewee.readthedocs.org/en/latest/peewee/models.html#creating-a-custom-field

However this technique will not work with SQLite, which looks for columns typed "INTEGER PRIMARY KEY" and will use these columns to replace the native `rowid`: http://www.sqlite.org/autoinc.html  -- Since the column we'd be adding would be BIGINT PRIMARY KEY Sqlite does not seem to auto-generate the ID (tested this locally).

Also, I'm also pretty sure that SQLite does not actually treat bigint any different than a regular integer.

This looks like it has some info on the topic: http://sqlite.1065341.n5.nabble.com/Impossible-to-declare-field-type-BIGINT-PRIMARY-KEY-td44651.html

If you are using something like Postgresql, you _can_ use a bigint, but you will need to create and specify a sequence to use to auto-generate the IDs.  Postgres will auto-create the sequence for `serial` columns, but if you use a bigint you will need to create the sequence by hand.
Issue: Incorrect SQL params in order_by expression
Author: jamorton
State: closed
Created: 2013-05-12 06:20:12

```
Player.select().order_by(Player.elo * 5)
```

printing the SQL:

```
('SELECT t1."id", t1."created_at", t1."last_modified", t1."wins_t", t1."wins_z", t1."wins_p", t1."losses_t", t1."losses_z", t1."losses_p", t1."username", t1."race", t1."elo" FROM "player" AS t1 ORDER BY (t1."elo" * ?)', [])
```

error:

```
Traceback (most recent call last):
  File "manage.py", line 43, in <module>
    func(sys.argv[2:])
  File "manage.py", line 25, in cmd_test
    for k in Player.winrate_order():
  File "/Users/jonanin/dev/sc2site/env/lib/python2.7/site-packages/peewee.py", line 1563, in __iter__
    return iter(self.execute())
  File "/Users/jonanin/dev/sc2site/env/lib/python2.7/site-packages/peewee.py", line 1556, in execute
    self._qr = ResultWrapper(self.model_class, self._execute(), query_meta)
  File "/Users/jonanin/dev/sc2site/env/lib/python2.7/site-packages/peewee.py", line 1314, in _execute
    return self.database.execute_sql(sql, params, self.require_commit)
  File "/Users/jonanin/dev/sc2site/env/lib/python2.7/site-packages/peewee.py", line 1721, in execute_sql
    res = cursor.execute(sql, params or ())
sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current statement uses 1, and there are 0 supplied.
```


Thanks!
Issue: No transactions support for MySQL
Author: haykinson
State: closed
Created: 2013-05-06 07:42:34

As far as I can tell, peewee just toggles autocommit on and off for transaction support. This is not really enough -- you need the START TRANSACTION / COMMIT / ROLLBACK commands issued too.


If you are using MySQL first be sure you are using InnoDB, as MyISAM does not support transactions:

http://dev.mysql.com/doc/refman/5.0/en/ansi-diff-transactions.html

Secondly, peewee will not explicitly execute 'begin', 'commit' SQL -- rather it will call the appropriate methods on the db driver.  For example `database.commit()`, `database.rollback()`.


which is much better for peewee transaction:
database.begin, database.commit()
vs
with database.transaction:


It doesn't matter, whichever you prefer.


Thanks! 
Another question:
did peewee place objects into cache for later use?
that means get a object from database at the first time and put it into cache, later will get from cache if it exists in cache. 


Peewee will cache the results of select queries so that multiple iterations do not cause multiple queries, but other than that there is no caching.

@bakerchen -- if you have questions, for future reference, please use either the [google group](http://groups.google.com/group/peewee-orm) or the #peewee channel on freenode.
Issue: Added an example for postgresql array type custom field in model docs 
Author: diffused
State: closed
Created: 2013-05-04 12:56:04

Thought this might help in the docs.


Dang, does that "just work" ?


So you are missing code to register the field with the database class -- db_field should be "textarray" and then you register the type "textarray" with "Text[]".

I tested this myself and got it working, but because peewee doesn't support (yet) any interesting array operations, I am not going to include this example for now as it might lead people to believe that peewee supports PG array operations.


Cheers for the tip. Yeah was pretty neat it just worked. 
Looking forward to the updates.
Issue: TypeError when inspecting a MySQL DB
Author: erkurita
State: closed
Created: 2013-05-03 23:31:43

I'm getting the following error when executing this command:

```
python pwiz.py -u root --password="<password>" -e mysql elecciones_2013 > ~/schema_elecciones.py
```

The error is:

```
Traceback (most recent call last):
  File "pwiz.py", line 417, in <module>
    print_models(options.engine, database, tables, **connect)
  File "pwiz.py", line 332, in print_models
    models, table_to_model, table_fks, col_meta = introspect(db, schema)
  File "pwiz.py", line 311, in introspect
    models[table][column][0] = ForeignKeyField
TypeError: 'tuple' object does not support item assignment
```

I changed the following line and it worked just fine. Here it is in diff:

```
diff --git i/pwiz.py w/pwiz.py
index d20ebf5..1ff2db7 100755
--- i/pwiz.py
+++ w/pwiz.py
@@ -171,7 +171,7 @@ class MySQLDB(DB):
     def get_columns(self, table):
         curs = self.conn.execute_sql('select * from `%s` limit 1' % table)
         return dict(
-            [r[0], (self.reverse_mapping.get(r[1], UnknownFieldType), r[6])]
+            [r[0], [self.reverse_mapping.get(r[1], UnknownFieldType), r[6]]]
             for r in curs.description)

     def get_foreign_keys(self, table):
```


Thanks for reporting!


@coleifer, I guess that's more elegant (and solves the problem too):

```
diff --git a/pwiz.py b/pwiz.py
index d20ebf5..5c61d21 100755
--- a/pwiz.py
+++ b/pwiz.py
@@ -308,8 +308,8 @@ def introspect(db, schema=None):
     for table in tables:
         col_meta[table] = {}
         for column, rel_table, rel_pk in table_fks[table]:
-            models[table][column][0] = ForeignKeyField
-            models[rel_table][rel_pk][0] = PrimaryKeyField
+            models[table][column] = (ForeignKeyField, models[table][column][1])
+            models[rel_table][rel_pk] = (PrimaryKeyField, models[rel_table][rel_pk][1])
             if rel_table == table:
                 ttm = "'self'"
             else:
```
Issue: TypeError when running a 'COPY' command on Postgres
Author: bwghughes
State: closed
Created: 2013-05-02 10:24:19

I'm loading a CSV file on Postgres using the COPY command, and get a TypeError (I think its because the COPY command returns None).

Here's the code:

``` python
def seed_towns():
    print "Seeding Towns tables..."
    towns_file = os.path.join(os.path.abspath(os.curdir), 'data/uk-towns-list/uk-towns.csv')
    Town.drop_table(fail_silently=True)
    Town.create_table(fail_silently=True)
    print "Loading data..."
    rq = RawQuery(Town, "COPY uk_towns FROM '{}' DELIMITERS ',' CSV HEADER".format(towns_file))
    try:
        rq.execute()
        # Wierdness here
   except TypeError, e:
        assert Town.select().count() == 43143
    finally:
        print 'Done.'
```

And here's the trace:

``` python
Traceback (most recent call last):
  File "./manage.py", line 45, in <module>
    manager.run()
  File "/Users/ben/repos/badbatch/lib/python2.7/site-packages/flask_script/__init__.py", line 351, in run
    result = self.handle(sys.argv[0], command, sys.argv[2:])
  File "/Users/ben/repos/badbatch/lib/python2.7/site-packages/flask_script/__init__.py", line 324, in handle
    return command.handle(app, *positional_args, **command_namespace.__dict__)
  File "/Users/ben/repos/badbatch/lib/python2.7/site-packages/flask_script/commands.py", line 143, in handle
    return self.run(*args, **kwargs)
  File "./manage.py", line 31, in seed_towns
    for obj in rq.execute():
  File "/Users/ben/repos/badbatch/lib/python2.7/site-packages/peewee.py", line 1365, in execute
    self._qr = ResultWrapper(self.model_class, self._execute(), None)
  File "/Users/ben/repos/badbatch/lib/python2.7/site-packages/peewee.py", line 1090, in __init__
    for i in range(len(self.cursor.description)):
TypeError: object of type 'NoneType' has no len()
```


Thanks for the excellent bug report.  I think I need to clarify the docs a bit -- `RawQuery` was meant to be used for expressing `SELECT` statements.  RawQuery is there to give you a way to specify any select statement and _return model instances_.

If you just want to execute some plain old SQL, try:

``` python

# assume somewhere you have a db defined
database = PostgresqlDatabase('something')

# ... later ...
sql = "COPY uk_towns FROM '{}' DELIMITERS ',' CSV HEADER".format(towns_file)
res = database.execute_sql(sql)
```


Grand - thanks for the clarification - its abit if an edge case. As I said before I lurrrrve Peewee Thanks !! :)) 


Thanks!  So glad to hear that.  I'm pushing an update to the docs in a few moments that should hopefully clear this up for the next person who runs into it.