akumoth/peewee-issues huggingface dataset:
Issue: group_by() broken for queries without aliases
Author: awahlig
State: closed
Created: 2011-12-30 23:03:54

This code:

``` python
from peewee import *

class Message(Model):
    author = CharField()
    body = TextField()

print Message.select().group_by('author').sql()
```

Throws this exception:

```
Traceback (most recent call last):
  File "test.py", line 7, in <module>
    print Message.select().group_by('author').sql()
  File "peewee.py", line 1216, in sql
    group_by = ', '.join(group_by)
TypeError: sequence item 0: expected string, tuple found
```

This part of SelectQuery.sql() seems responsible:

``` python
        if self.use_aliases():
            (...)
        else:
            group_by = [c[1] for c in self._group_by]
```

c[1] is a sequence of field names, not a single field name, so str.join() fails. The code used when use_aliases() returns True seems to work better.
Issue: Auto save() dirty objects on commit
Author: coderbuzz
State: closed
Created: 2011-12-30 10:49:18

Hi, to speed-up my query tasks, by default I have turned-off autocommit to False in db configuration, 
means we need to call db.commit() to commit the transaction

Found now behavior doesn't flush (save) dirty objects on commit. 
Again, in sql-alchemy we don't need explicit call object.save() as we know there's dirty objects that need to be save() automatically on commit.

``` python
try:
    othermodel.create(blablafield = 'blabla value')

    mymodel = MyModel.select().first() # hehe new stuff :D
    mymodel.somefield = 'edit the field value'  # dirty object
    # mymodel.save() # maybe explicitly required only on autocommit scheme

    db.commit()
except :
    db.rollback()
```

I'm playing with more than 4 models in one task, which changing the field value anywhere,
it's a bit difficult to catch which object should call save() on the end of procedure block since changes could be applied on several place of code block logic


If you take a look at the testcases it shows how things are saved:

https://github.com/coleifer/peewee/blob/master/tests.py#L2702

Basic workflow should be:
- turn off autocommit
- modify a bunch of models or whatever
- call `.save()` on the models you want to save
- call `.commit()` on the database and all the saves (update/insert) will be committed

The test is showing:
1. turn off autocommit
2. save two blogs
3. open up a new connection to the same database (`new_db`)
4. using that new connection query to see if there are any blogs
5. assert that there are none
6. call commit on our original connection (`test_db`)
7. use the other connection to query again for blogs
8. assert that they are present

There is no "automatic" saving of dirty models.
Issue: Column default value wont (keep) update
Author: coderbuzz
State: closed
Created: 2011-12-28 14:42:47

Hi, I have the following model which have a datetime field with default value=datetime.now()

``` python
from datetime import datetime

class ATable(BaseModel):
    time = DateTimeField(default=datetime.now())
    customer = CharField()
```

Surprised when I create several records say:

``` python
ATable.create(customer='A')
# some delay
ATable.create(customer='B')
# some delay
ATable.create(customer='C')
```

The time (default) filled values was all the same: 2011-12-28 21:30:38.257 on all records
The time beeween record creation surelly in different time, at least microsecond and or second value

Perhaps a caching issue?

Thanks


Dooh my mistake, should be:

``` python
time = DateTimeField(default=datetime.now)
```

In previous I pass the now() value, not passing function as arg.

Sorry to pollute
Issue: Field allow null by default
Author: coderbuzz
State: closed
Created: 2011-12-25 18:42:12

Hi, when initializing (create) record; sometimes; mostly we have to leave other non indexed fields with default value or null for future fill.

May I suggest you to allow null by default, it's rare of important field to be not null

``` python
class Field(object):
    def __init__(self, null=True, db_index=False, unique=False, verbose_name=None,
                 help_text=None, *args, **kwargs):
```

null=**True**

Thanks


Thanks for the suggestion -- I am going to stick with the current behavior, though.
Issue: QueryResultWrapper.first()
Author: coderbuzz
State: closed
Created: 2011-12-25 15:13:08

get() is not suitable in some cases, for example if we need get first row from complex select-where query, using join, sub-query etc (without exception)

Can you please add a first() or using get() name too method to QueryResultWrapper class?
The method should return None on empty resultset, otherwise return first row

Please check of mind:

``` python
class QueryResultWrapper(object):
# ...
    #def get(self): or which suitable
    def first(self):
        if self._result_cache:
            return self._result_cache[0]
        else:
            row = self.cursor.fetchone()
            if row:
                row_dict = self._row_to_dict(row, self.cursor)
                instance = self.model_from_rowset(self.model, row_dict)
                self._result_cache.append(instance)
                return instance
            else:
                self._populated = True
                return None
```


Good points...I will work on adding this -- the problem w/the implementation you suggest is that a subsequent iteration will start on the 2nd result.


Thank you! :)
Issue: Feature, blob field.
Author: jander
State: closed
Created: 2011-12-19 13:49:31

i want to save image in database by peewee.
but peewee has not BlobField yet.


I haven't done a _ton_ of research on this issue, but it should be possible to add a custom field class with the proper column type.  I've added 'blob' as a column type so you can specify `db_field = 'blob'` and it will create the right column.  Looks like with sqlite3 and psycopg2 at least you'll need to wrap the incoming data using `sqlite.Binary` or `psycopg2.Binary` http://stackoverflow.com/questions/3379166/writing-blob-from-sqlite-to-file-using-python and http://initd.org/psycopg/docs/usage.html#adaptation-of-python-values-to-sql-types ... There are also questions as to the proper python type to return.  Since peewee gives you the ability to create custom fields, and because of the apparent complexity, I'm going to close this issue and suggest that users who need this functionality roll their own specific to their use-case.  Sqlite, for example, returns a python buffer object.  Here's an example implementation for sqlite3:

```
class BlobField(Field):
    db_field = 'blob'

    def db_value(self, v):
        return sqlite3.Binary(v)
```

Alternatively, you could use a TextField and just convert data from/to base64 when storing in the database.


this is not working on 2.0.9 version. I receive a:
KeyError: 'blob'

I added 'blob': 'BLOB', after line 605.


@trunet, my apologies, I should have posted an update.  This is no longer included by default in peewee.

It shouldn't be difficult to add your own implementation and it might be something I will add down the road:

http://peewee.readthedocs.org/en/latest/peewee/models.html#creating-a-custom-field


@trunet -- I added a blob field in 3cabdf3f7139cf9283d74bd27a2399ec94a84551


nice, thank you very much.
Issue: UpdateQuery need more?
Author: jander
State: closed
Created: 2011-12-19 13:45:31

A simple example:

Model class

``` python
class Entity(Model):
    num = IntegerField(default=0)
```

peewee seems no way for sql:

```
update Entity set num=num+1 where num>?
```


Yes, totally agree this would be good.  Thinking about an implementation.
Issue: Behaviour of __in against foreign keys doesn't behave quite like one might expect
Author: chadcatlett
State: closed
Created: 2011-12-14 22:25:34

Given the following models:

``` python
class ModelA(model):
    blah  = CharField()

class ModelB(model):
    modela = ForeignKeyField(ModelA)
    blah  = CharField()
```

Doing a query like this, gives results not quite expected:

``` python
modb = ModelB.get(id=1)
modas = ModelA.select().where(id__in=modb.modela)
```

The second query uses ModelB's primary key, instead of the modela_id column. The above example is sort of useless, as one could just use modb.modela, but when combined with Q():

``` python
modb = ModelB.get(id=1)
modas = ModelA.select().where(~Q(id__in=modb.modela))
```

The above would allow one to fetch all ModelA records that aren't related to ModelB.

Changing this would break the current behavior, but in my opinion it would make it behave more like the code actually reads.

What do you think?


I think there are some problems with your examples -- in the first you're doing an __in on a single model instance -- modb.modela is a single model a associated with the given modb.  YOu could rewrite as:

```
moda = modb.modela
```

In the second, you want modela records with that aren't related to some given modb... I _believe_ this should work:

```
modas = ModelA.select().join(ModelB).where(~Q(id=modb.id))
```


OK -- closing for now, reopen if you get more info.
Issue: Support for pysqlite
Author: awahlig
State: closed
Created: 2011-12-14 18:09:07

To my knowledge, the sqlite3 module from the stdlib has the same API as the external pysqlite:
http://code.google.com/p/pysqlite

In case the sqlite3 module is absent, peewee could try to use pysqlite instead:

``` python
try:
    import sqlite3
except ImportError:
    try:
        from pysqlite2 import dbapi2 as sqlite3
    except ImportError:
        sqlite3 = None
```


sqlite3 was added in 2.5 and since peewee is 2.5+ compatible, there's really no need.  If someone should want to use the alternate driver they can simply subclass the SqliteAdapter/Database and override as needed.  Thanks for bringing this up, though!
Issue: Small fixes and simplification
Author: lukaszb
State: closed
Created: 2011-12-13 21:35:55

I've removed some not used imports, changed test runner a little bit (so it can actually run single test, i.e.:

```
./runtest.py QueryTests.test_count
```

Fixed 2 (out of 3) failing tests. Last one (_QueryTests.test_count_) ... well, tried to debug it but the problem seem to be somewhere during iteration of _SelectQuery(Blog)_, not sure where exactly.


Cool, will take a look


The two test failures are relted to non-deterministic ordering.. I need to work up a real fix for those.

I cherry-picked 9c12013f48d447584584781114f59a8fc7ca126f however, which was your fixes to the test runner.  Thanks a ton!
Issue: Features/allow custom commit settings
Author: deytao
State: closed
Created: 2011-12-11 13:17:55

Hi!

I was a lil bit confused that all my insert and update was commit immediately.
Sometime I need to insert several records at once, I saw a rollback method but it was never used.
I decided to add a commit arg in the save method.
It was the simplest and fastest way to fix my problem.

Maybe I missed something.


Thanks for bringing up this issue -- I've added per-connection methods to toggle autocommit, letting you run multiple queries in a single transaction.  The changeset is here b46ede9548a80ce2a9f135404f39cd363b6ba03d

You can check the API in the test cases above, but looks like:

```
my_database.set_autocommit(False) # turn it off
insert_a_bunch_of_rows()
my_database.commit() # they are all committed in a single transaction
```

Also:

```
@my_database.commit_on_success
def some_function():
    insert_a_bunch_of_rows()
    insert_some_bad_data_and_die()
```

The above will issue a rollback and then raise the exception, and the rows inserted by "insert_a_bunch_of_rows" will be rolled back as well.


That's perfect! Thanks!
Issue: Allow custom sequence name
Author: deytao
State: closed
Created: 2011-12-03 18:02:07

- Use Meta class to set it like
  
  ``` python
  class Meta:
      pk_sequence = 'sequence_name'
  
  ```
  
  Helpful when using a global sequence.


Cool, at first glance it looks good.  We'll need to ensure that if a pk_sequence is specified that the column is created with that sequence as well.  Lastly, do you think this is something that should be inheritable from model to model?  i.e., the database Meta attribute is inheritable, but the db_table is not.


I've added the "read" part of this feature in 22639933f3079e6f01be173e5637672ce14f35cf but have not come up with the correct approach for "write", i.e. table creation.  Will be thinking about this, though for now should wokr.


It should be inheritable because it has two goals.
- To let the developer set his own sequence name for each table
- To allow one sequence for all table or a group of table, like table inheritance in postgresql

I think we shouldn't allow a pk_sequence attribute in children Meta.
To lock this behavior maybe it could be interesting to raise an exception. But it could also be possible to avoid the check since the parent is found.


Yeah, I should've updated my notes.  I've added logic to create sequences automatically when running `Model.create_table()` and to share them across models using inheritance.  It's in commit cc5d86f15dee40d45a0eacc95d346175c40eb99e

You can run the test suite using postgres:

```
createdb peewee_test
export PEEWEE_TEST_BACKEND=postgresql
python setup.py test
```
Issue: Selecting databases in Meta is very non-flexible
Author: jamorton
State: closed
Created: 2011-11-28 01:33:08

I don't know Django very well, but as far as I could tell, Django doesn't use the Meta class in models to select the database to use, and probably for a good reason?

I really like peewee, but I had to restructure some of my base application flow because all Peewee model classes have to have the database instance available immediately on class creation and in some global scope. The examples of course look fine, but for an application like mine that is larger and uses blueprints and all that, the Meta/inheritance system was a bit awkward to use.

do you think there's a way to just have a global bind variable? Like Elixir:

```
 elixir.metadata.bind = "sqlite:///db.sqlite"
```

Put simply, it would be nice to have a way to set the database object after the Model classes have been created.


Thanks for bringing this up - the need to have a database instance at model definition time is ... well, it is what it is.  You brought up django -- django declares the database(s) in a global (and much maligned) settings object.  One of the decisions I made with peewee was to make it _not_ need global database configuration.  One question is, if you don't declare your database at class creation time, what time _is_ appropriate?

I'm not sure if this will help you, but my general workflow looks like this (to avoid circular imports):
- app.py
  - flask application instantiation
  - peewee database instantiation
  - create a "BaseModel" class that uses the above database
- models.py or whatever
  - import the BaseModel from app.py and extend for any specific models I need
- views.py
  - import the flask app from app.py to create blueprint/url rules/etc
  - import any models needed from app.py
- main.py
  - import the flask app from app.py
  - import all the views so they get registered
  - run the flask app or whatever


I think that database selection should be be able to take place any time before a connection needs to be made, so in the case of flask, sometime before @app.before_request ideally. For example, instead of the model classes holding a pointer to a database object, they hold a pointer to a callable that will give them a database object when they're instantiated. Or, both are permitted so the simple case doesn't change.

I understand that there are reasonable ways to do everything in the global scope - and maybe that's the pythonic way - but it seemed weird when I had to move a lot of my stuff from classes into toplevel code to get it working.


Cool, also bear in mind that there's a difference between the database object (what you use with models) and the underlying connection/when that connection is made.  It is made when it is first needed if not explicitly created.  Also, you may want to check out flask-peewee, which does some of the things you've mentioned.

https://github.com/coleifer/flask-peewee/tree/master/example
Issue: Investigate selecting related fields
Author: coleifer
State: closed
Created: 2011-11-25 20:05:33

Would be nice to be able to populate (or partially populate) instances of related objects when querying.  Example would be querying a list of blog entries and wanting to additionally select the 'name' of the blog -- this can be done in 1 query with a join.

It shouldn't be too hard... related models can be queried passing a dictionary to Model.select() -- need to add the logic to the QueryResultWrapper.  Looking at the returned rows and converting them into a dictionary may not work here if column names collide, so might need to list the columns and assign based on index.

Now that fields are exposed using descriptors it sohuld be possible to do "deferred" loading of field data as well, i.e. if it wasn't selected but then is asked for.


That looks wonderful.


I added support for this in 62bd00b90686aea6d6440ff129c30cf28a2a9449 -- basically, it takes what you pass in to the select() method, queries the db, constructs instances (and related instances), then puts them all together.  Can reduce your queries from _n_ to 1 when iterating and checking up the fk tree.  It doesn't go 'down' the chain, though...ie you can't select Blog and then preload their related entries, though implementing this in python is not hard.

From the test cases:

```
def test_select_related_multiple(self):
    qc1 = len(self.queries())

    sq = EntryTag.select({
        EntryTag: ['*'],
        Entry: ['pk', 'blog_id'],
        Blog: ['title'],
    }).join(Entry).join(Blog).where(title='b1')

    results = list(sq)
    blog_titles = [r.entry.blog.title for r in results]
    blog_ids = [r.entry.blog.id for r in results]

    self.assertEqual(blog_titles, ['b1', 'b1', 'b1'])
    self.assertEqual(blog_ids, [self.b1.id, self.b1.id, self.b1.id])

    qc2 = len(self.queries())

    self.assertEqual(qc2 - qc1, 1)
```
Issue: [Postgresql] Boolean data type
Author: deytao
State: closed
Created: 2011-11-23 16:42:24

If I create manually my table with a boolean type I can't insert a record using a field like BooleanField because it try to insert an integer.
Postgresql handles correctly bool() and stores a proper boolean field not like MySQL or Oracle which are casting in integer.


Fixed in 31b213e4db37f39fe3374ad2ea4143350e126242 - tested against postgresql, mysql and sqlite
Issue: Add field inheritance
Author: deytao
State: closed
Created: 2011-11-23 16:38:56

Hi,

It would be nice if something like that could work

``` python
import peewee


Class CoreModel(peewee.Model):

    id = peewee.PrimaryKeyField()
    created = peewee.DatetimeField()

    Class Meta:

         database = db


Class User(CoreModel):

    firstname = peewee.CharField()


Class Car(CoreModel):

    brand = peewee.CharField()


```

I don't want to duplicate my fields.
Also for the id when I'm using a global ID.


Totally -- creating models should be DRY.  This should work and currently does not (need to add some logic in the metaclass).  I'm not sure what you mean by using a global ID, but multi-table inheritance is not something I plan on implementing.  If you're looking for that level of control, definitely would recommend http://www.sqlalchemy.org/docs/orm/inheritance.html


I saw you were creating a sequence for each primary key. Which is correct.
But with my proposition I was hoping to get one sequence for all tables inheriting of the "coremodel"
I try to implement it, but my python skills are def not that good.
Reading your code taught me a lot about python, but not enough to fork it.

SQLAlchemy is too heavy, I don't really need it for my project.


I've fixed the bug where fields are not inherited.  As for the part about sharing a sequence, that is not something that will be implemented as part of peewee.
Issue: Add support for bigint and double fields
Author: marmelo
State: closed
Created: 2011-11-18 02:38:13

Currently you can only define IntegerField, FloatField and DecimalField.

What happens if you need to store a long (bigint)?
You have to use a Decimal without decimal places?
A new **LongField** (or BigIntField) should be added (**bigint, 8 bytes**).

Also, sometimes float is not enough.
For coherence, **DoubleField** should also be added (**double precision, 8 bytes**).

I believe both cases are extremely frequent.


This will become handy. Thanks!
Issue: FloatField is translated differently between supported implementations
Author: marmelo
State: closed
Created: 2011-11-18 02:29:38

Floats, by definition, should only take 4 bytes.

_Currently:_
- SQLite, real, 8 bytes
- MySQL, double precision, 8 bytes
- PostgreSQL, real, 4 bytes

_Proposal:_
- SQLite should continue using real (I believe there is no 4 bytes floating point datatype).
- MySQL should be using float, 4 bytes.


Thanks.
Issue: New Model Defaults
Author: jerodsanto
State: closed
Created: 2011-11-15 16:32:14

When you create a model that has default field values they get used when persisting to the db (and subsequently when being initialized from the db) but not when a new object is instantiated. A contrived example:

``` python
class Post(peewee.Model):
    comment_count = peewee.IntegerField(default=0)

post = Post()

post.comment_count # => None
post.save()
post.comment_count # => 0
```

I think it'd be more consistent if the defaults were applied to all objects of the class, not just ones that have been persisted or come from the database.

The change is quite simple, I believe. We just need to modify the `Model` class's `__init__` method like so:

``` python
def __init__(self, *args, **kwargs):
    self.get_field_dict()
    for k, v in kwargs.items():
        setattr(self, k, v)
```

If this is something you'd be willing to merge, I'll happily do the work and submit a pull request.

Let me know, thanks.
Issue: add "is not null" lookup support
Author: jerodsanto
State: closed
Created: 2011-10-03 17:58:48

You have a way to query for columns with NULL values but not NOT NULL values, so I added it.

I wasn't sure if you'd prefer `__isnt` or  `__is_not`, but I went with the latter because it's more explicit.


I'm not going to merge this since you can accomplish the same using `Q` objects:

```
NullModel.select().where(~Q(char_field__is=None))
```

I added this to the testcase to show how it is possible:
https://github.com/coleifer/peewee/commit/08dabe0d6b82701c48022e5d7a758d29bfaea154


Fair enough, though you could make the same case for the `ne` operator.


Q objects don't appear to exist in my version of Peewee. How can I accomplish the same behavior without them? I have models like:

class Owner(Model):
    ...
class Ownable(Model):
    owner = ForeignKeyField(Owner, null=True, default=None)

And I wish to obtain a list of all Ownables which do not have an owner.


You would do `Ownable.select().where(Ownable.owner >> None)` -- if you want not null: `~(Ownable.owner >> None)`
Issue: Q() objects cannot be mixed between models
Author: coleifer
State: closed
Created: 2011-09-14 14:31:24

It is not possible to create an "OR" query comparing columns from two tables.  This is due to the way where clauses are created programmatically:

```
# where clauses between models are separated by calls to join() or switch()
Blog.select().where(Q( ??? )).join(Entry).where( Q(???) )
```

With the new .filter() syntax, it is syntactically possible to create these types of queries:

```
Blog.filter(Q(title='my blog') | Q(entry__title='some special entry'))
```

But there is no underlying mechanism to generate the right SQL since where clauses are stored in a dictionary keyed by the model they pertain to.

Fixing this will most likely require significant rewriting.


This has been fixed in 3f6336bba6d3a0bda691fe3e333767008c2308bb