akumoth/peewee-issues huggingface dataset:
Issue: Peewee/psychopg2 always in transaction mode
Author: RickMeasham
State: closed
Created: 2020-06-16 06:22:38

When there's a database error, postgres leaves me in a failed transaction. This appears to be because psycopg2 [defaults to manual commits](https://www.psycopg.org/docs/connection.html#connection.autocommit). So I updated the connection to pass through `autocommit=True` but it appears I'm still stuck in failed transactions when there are errors:

Here's my connection code:
```
database = PostgresqlExtDatabase(
    config.database('dbname'),
    host=config.database('host'),
    user=config.database('username'),
    password=config.database('password'),
    autocommit=True
)
```
Is that the right way to connect with autocommit?

Here's the traceback if it helps:

```
Traceback (most recent call last):
  File "/var/task/handler/trial.py", line 97, in handler
    return trial_new(event)
  File "/var/task/handler/trial.py", line 22, in trial_new
    client = User.get(User.mobile == mobile)
  File "/var/task/peewee.py", line 6384, in get
    return sq.get()
  File "/var/task/peewee.py", line 6807, in get
    return clone.execute(database)[0]
  File "/var/task/peewee.py", line 1886, in inner
    return method(self, database, *args, **kwargs)
  File "/var/task/peewee.py", line 1957, in execute
    return self._execute(database)
  File "/var/task/peewee.py", line 2129, in _execute
    cursor = database.execute(self)
  File "/var/task/playhouse/postgres_ext.py", line 490, in execute
    cursor = self.execute_sql(sql, params, commit=commit)
  File "/var/task/peewee.py", line 3106, in execute_sql
    self.commit()
  File "/var/task/peewee.py", line 2873, in __exit__
    reraise(new_type, new_type(exc_value, *exc_args), traceback)
  File "/var/task/peewee.py", line 183, in reraise
    raise value.with_traceback(tb)
  File "/var/task/peewee.py", line 3099, in execute_sql
    cursor.execute(sql, params or ())
```

>When there's a database error, postgres leaves me in a failed transaction.

Yes, in which case you should issue a rollback.

Peewee is currently designed to work *with* psycopg2's transaction handling. Setting `autocommit=True` may break Peewee's transactional APIs or may have other unintended consequences. Unless you're very familiar with both peewee's transaction semantics and psycopg2's, I wouldn't recommend setting pg to autocommit.

If the issue is simply that you are issuing queries that fail due to errors, I suggest you fix your queries to make them more robust, and wrap them with peewee's `atomic` (or ensure that you properly roll-back upon a query failure).

By the way, your traceback is not relevant I don't think, since that is the query you are attempting to execute *after* a previous query failed.
Issue: How to scrape an img src url?
Author: xdebbie
State: closed
Created: 2020-06-13 17:25:50

I'm using peewee to scrape a webpage, and everything is working fine except when I try to get the url of an "img src=" tag. Does anyone know what is the correct syntax to get the url?

My code goes like this:
``` 
    # record label
    label = resp.html.find('.labels-list__item')
    if label:
        data['label'] = label[0].text

    # genre
    genre = resp.html.find('.genre-list__link')
    if genre:
        data['genre'] = genre[0].text

    # artwork
    artwork = resp.html.find('.single-album-tombstone__art img')
    if artwork:
        data['artwork'] = artwork[0].text

```

The # record label and # genre tags work fine, but with the #artwork it returns an "EMPTY" tag on my database. This is the html:

![website](https://i.imgur.com/vzorTL6.png)

How can I insert the attribute 'src' so it will return the actual url of the image? 



This is a question about some scraping library you are using. peewee issue tracker is not for questions, much less questions about a completely unrelated and unidentified html library.
Issue: model to dict extra_attrs
Author: JPilarr
State: closed
Created: 2020-06-11 22:39:15

Hello,
i have model like.
```
class Printer(pw.Model, ch.Model):
    # DB fields
    id = pw.AutoField(primary_key=True)
    name = pw.CharField()

    # cache field stored in redis
    state = ch.CharField(default=State.OFFLINE.name)
```
i have problem serialize redis part of model by model_to_dict. If i use exclude it work's but  if i use only and after use something like result['state'] = self.state. it doesn't work with error `"TypeError: Object of type CharField is not JSON serializable"`
```

        self.cache_get()
        self.fix_state()

        result = model_to_dict(self, only=[Printer.id, Printer.name,
                                           Printer.location, Printer.state]
                               )
        # from json import load
        result['state'] = self.state

        if self.printer_type_id:
            result['printer_type_id'] = self.printer_type.id

        return result
```

can you help me whats wrong? I dont' want use huge exclude. Is it way how to do it by extra_attrs parameter for model_to_dict.

I'm sorry but I don't understand the question, what these special field subclasses are, or really what the problem is.
Issue: Should AsIs() set converter attribute to False?
Author: theosotr
State: closed
Created: 2020-06-10 15:56:10

According to the documentation of `AsIs`.

> Represents a Value that is treated as-is, and passed directly back to the database driver. This may be useful if you are using database extensions that accept native Python data-types and you do not wish Peewee to impose any handling of the values.

However, this is not true, as `peewee` impose handling of the value passed in `AsIs`on this line https://github.com/coleifer/peewee/blob/master/peewee.py#L622

Should `AsIs` initialize a `Value` object as following?

```python
def AsIs(value):
    return Value(value, converter=False, unpack=False)
```

One may still want the option to apply converter logic, so converter is not explicitly disabled.
Issue: sub-expressions not wrapped by parentheses
Author: theosotr
State: closed
Created: 2020-06-09 09:33:53

I have the following query (simplified for opening issue):

```python
expr = (Value(1) + Model.column)
squared = (expr * expr)
Model.select(fn.sum(expr), fn.avg(squared)).objects()
```

peewee generates the SQL query below

```sql
SELECT sum(1 + "t1"."column"), avg(1 + "t1"."column" * 1 + "t1"."column") FROM "model" AS "t1"
```

Notably, the sub-expressions in `avg(1 + "t1"."column" * 1 + "t1"."column")` are not wrapped by parentheses as expected. That is

```sql
SELECT sum(1 + "t1"."column"), avg((1 + "t1"."column") * (1 + "t1"."column")) FROM "model" AS "t1"
```

Thanks for reporting. This bug was caused by modifying an internal flag on the expression node without copying it first. This caused subsequent usages of that expression node to carry-over the flag when they shouldn't have.
Issue: Database.bind_ctx preserves the binded database
Author: cslanziano
State: closed
Created: 2020-06-09 02:35:09

I am trying to set the database at run-time. Then the Models do not have the meta class defined. That means `Model._meta.database` is `None`.  The first code below executes as I expected, after the `with` block the database parameter return to `None`. 
```python
from peewee import SqliteDatabase, Model

class TableA(Model):
    pass
class TableB(Model):
    pass

db = SqliteDatabase(':memory:')

print('before', TableA._meta.database)
with db.bind_ctx([TableA, TableB]):
    print('inside A', TableA._meta.database)
    print('inside B', TableB._meta.database)
print('after', TableA._meta.database)
```

The issue is when a third table has foreign keys to both tables. The `Model._meta.database` is not returned to `None` as I was expecting. 

```python
from peewee import SqliteDatabase, ForeignKeyField, Model

class TableA(Model):
    pass

class TableB(Model):
    pass

class TableAB(Model):
    field1 = ForeignKeyField(TableA)
    field2 = ForeignKeyField(TableB)

db = SqliteDatabase(':memory:')

print('before', TableA._meta.database)
with db.bind_ctx([TableA, TableB]):
    print('inside A', TableA._meta.database)
    
print('after', TableA._meta.database)
```

The `bind_backrefs` set to `False` solves the issue. But, is this a bug? or is the normal behavior?.  The version of peewee is `3.13.3`.



Thanks for reporting. This is a bug and has to do with peewee performing redundant trips through the model graph.
Issue: Postgresql materialized views not showing
Author: albireox
State: closed
Created: 2020-06-07 17:26:40

It seems Postgresql materialized views are not seen as a table or a view in Peewee. I have a model of a materialized view that works fine otherwise but `Model.table_exists()` returns `False`. If I try to find it in `database.get_tables()` or `database.get_views()` it does not appears anywhere. Is there a way to check if a materialized view model actually exists?

First result on google my dude.

https://stackoverflow.com/questions/58720894/check-a-materialized-view-exists

I know how to use the `pg_class` table, "my dude", I was asking if this can be do directly in Peewee using its API. I won't be opening any more issues trying to improve Peewee since there doesn't seem to be an interest for it.
Issue: SUM subquery
Author: Feijo
State: closed
Created: 2020-06-05 03:24:17

I am working on a formula-based application, which each function contructs a piece of the SQL query that will be executed. In other words, each function is a formula, which can call other formulas.

For example:

```
class Formulas():
    def formula_piece_1(self, i, z):
        return (MyEntity.select(MyEntity.value_1).where((MyEntity.ClientId == i) | (MyEntity.date == z))).alias('subquery')

    def formula_piece_2(self, i, j):
        return MyEntity.select(fn.MAX(self.formula_piece_1(i, j).select(MyEntity.value_1)))

    # I'd have thousands of functions like these, that could be combined in any order. 

f = Formulas()
result = f.formula_piece_2(12, date(2016, 1, 1))
```

This is the query I was expecting:

```
SELECT MAX (subquery.value_1)
FROM (
  SELECT "t1"."value_1"
           FROM "my_entity" AS "t1"
           WHERE (("t1"."client_id" = 12) OR ("t1"."date" = '2016-01-01'))
) subquery
```

Instead I got this:
```
SELECT MAX
       (
         SELECT "t1"."value_1"
         FROM "my_entity" AS "t1"
         WHERE (("t1"."client_id" = 12) OR
               ("t1"."date" = '2016-01-01'))
       )
FROM "my_entity" AS "t1"
```

What would be the correct way of doing it? 

```python

subq = (MyEntity
        .select(MyEntity.value_1)
        .where((MyEntity.ClientId == i) | (MyEntity.date == z)))
        .alias('subquery'))
MyEntity.select(fn.MAX(subq.c.value_1)).from_(subq)
```
Issue: Regarding aliased columns of relationships
Author: theosotr
State: closed
Created: 2020-06-02 08:14:50

I have the following models

```python
class Author(BaseModel):
    first_name = CharField()
    surname = CharField()

    class Meta:
        table_name = 'author'

class Book(BaseModel):
    author = ForeignKeyField(column_name='author_id', field='id', model=Author)
    isbn = CharField(null=True, unique=True)
    title = CharField()

    class Meta:
        table_name = 'book'

class Review(BaseModel):
    book = ForeignKeyField(column_name='book_id', field='id', model=Book)
    content = CharField(null=True)
    rating = IntegerField(null=True)
    reviewer_name = CharField(null=True)

    class Meta:
        table_name = 'review'
```

And I perform the following query

```python
name = Review.book.author.first_name.alias('name')
res = Review.select(name).join(Book).join(Author).where(
    (Review.rating >= 2) & (Author.first_name <= Value('Z')))
for r in res:
    print(r.name)
```

this gives me the following error

```
Traceback (most recent call last):
  File "test.py", line 10, in <module>
    print(r.name)
AttributeError: 'Review' object has no attribute 'name'
```

To make the code above to work, I have to print `r.book.author.name`.
When I create compound aliased column (e.g., (`Review.book.author.first_name + Review.book.title).alias('name')`), the result set *does* contain the property `name` as expected.

For example, this query works

```python
name = (Review.book.author.first_name + Review.book.title).alias('name')
res = Review.select(name).join(Book).join(Author).where(
    (Review.rating >= 2) & (Author.first_name <= Value('Z')))
for r in res:
    print(r.name)
```

I don't know if this is a feature or bug, but I find it counter-intuitive. I would expect peewee to store all aliased columns to the object of the result set, i.e., I would like to make an alias on the namespace of the final result set. Am I missing something here?

Peewee puts stuff on the model it belongs to. You're just aliasing the author's first-name, so peewee puts that on the author model. When you have an expression where it is unclear what model it belongs to, then it gets placed on the root/selected model.

If you don't want to be dealing with a graph of models, just use `.objects()` at the end of your query and *all* attributes will be placed on the root/selected model.
Issue: According to the MySQL worklog, VALUES function will be deprecated soon
Author: ShuFFel
State: closed
Created: 2020-06-01 13:14:00

Found task in MySQL worklog: [https://dev.mysql.com/worklog/task/?id=13325]
Right now it's just a warning about deprecation, but seems that this function will be removed in future

Thanks for the heads-up. I wonder how they intend to reference the selected column. Anyways we'll cross that bridge when we get there.
Issue: any plan to support async/await
Author: DieterLi
State: closed
Created: 2020-05-25 04:10:46

as the title. Thanks, team

Discussed and dismissed many times, notably #263. There are so many things I dislike about the ~twisted~ asyncio approach, mostly the fact that you have to write "asyncio"-specific everything -- drivers, ORMs, application frameworks, etc., etc. and even then you still have a "[what color is my function](https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/)" situation.

Use gevent. Seriously. You can thank me later.

A commenter suggested [tortoise orm](https://tortoise-orm.readthedocs.io/en/latest/) - I cannot vouch for it, but wanted to preserve the recommendation for anyone who arrives here via google.

@coleifer it's very sad that you ignore the `async/await` syntax. A lot of people/products use it these days and it's not only about `asyncio`. I know about the tortoise ORM, but I prefer Peewee syntax.

Maybe we have a chance to split the Peewee ORM and make a code that working with DB connections independently. So there will be drivers independent Peewee part which one only is responsible for the conversation Models <-> SQL and another part that implement database connection management. That way people could be available to use Peewee with any sync/async drivers.

@klen - I wouldn't say that I ignore it, but I do not like the approach (for a number of reasons) and prefer the gevent approach for Python. The main reason I dislike it is because it requires a special "asyncio-aware" version of every piece of software in the stack.

This project exists, which might prove useful for people finding this issue in the future:

https://github.com/05bit/peewee-async

As an alternative, there is another library: https://github.com/klen/peewee-aio
Issue: Request: make table_function to be evaluated during first time Model being instantiated.
Author: futursolo
State: closed
Created: 2020-05-23 12:27:01

Rationale
---------
Currently, `table_function` is evaluated as soon as a `Model` class is defined.

However, just like`peewee.DatabaseProxy()`, sometimes table prefix is also not known until the config file is parsed.

Current Workaround
--------------------
I carefully checked imports to make sure no database models are loaded until config file is parsed. But it is kind of difficult to keep track of files as the code base grows. And this separates the program into two separate parts `config` and `main[web handlers, database_models]` which is less than ideal.

Potential Solutions
-------------------
1. make `table_function` to be evaluated during first time `Model` being instantiated.

2. let `Database` carry a `prefix` attribute and peewee read it every time it tries to generate a SQL statement.



Yes, we evaluate it at class-creation time. I'm going to pass on the solutions proposed, as they seem rather specific to whatever you're doing.
Issue: Postgres TimeFields with timezones
Author: BryceBeagle
State: closed
Created: 2020-05-20 06:46:54

`DateTimeField` has a timezone-based subclass in `postgres_ext.DateTimeTZField`. Is there a reason why there's no analog for `TimeField` (i.e. `TimeTZField`)?

Also related, the [docs](https://github.com/coleifer/peewee/blame/master/docs/peewee/playhouse.rst#L688) mention (and try linking to) `DateTimeTZ` (as opposed to `DateTimeTZField`) resulting in a broken link

How can a time have a timezone, when the time's value is dependent on the date it's attached to? At any rate, you can subclass `TimeField` and provide your own field-type. Creating field types is super easy, especially when they are derived from existing field.

Thanks for the note about the docs, I've fixed it.

Postgres has a [datatype that's just a time](https://www.postgresql.org/docs/current/datatype-datetime.html) (no date) with a timezone. This makes sense because I can say "7PM EST" and convert it to UTC without ever needing to get a date involved.

Python's `datetime.time` also supports timezones with its `tzinfo` attribute.

I understand if you don't want to add it, and can go with the custom field, but to me it seems strange to have first class support for the `timestamp with time zone` datatype but not the `time with time zone` one.
Issue: Can't use Model class as field value
Author: mikemill
State: closed
Created: 2020-05-19 18:04:54

I have a custom field that stores the import path to a Python class.  The `db_value` takes the class object and coverts it to the import path string and the `python_value` takes the string and imports the object giving me back the class.  However, as of peewee 3.13.2 this no longer works and instead gives me `peewee.InternalError: (1054, "Unknown column 't1' in 'field list'")` when I try to save.

Field:
```python
class PythonObjectField(CharField):
    """
    A custom class for storing a Python object as a reference.
    """

    def db_value(self, value):
        if value is None:
            return None
        elif isinstance(value, str):
            return value

        return get_object_path(value)

    def python_value(self, value):
        if value is None:
            return None

        return import_object(value)
```

The usage would be `Model.field = SomeOtherModel`

[Full example script](https://github.com/coleifer/peewee/files/4652313/peewee_example.txt)

When debugging this I found that this affects my `Model` derived classes but not a simple one that only extends `object`.  Additionally, I found that `db_value` is never called.



It isn't calling `db_value()` because peewee sees that you're using a model class as a value, which is a recognized AST type. The issue that originated this fix is #2131.

Thanks for reporting, this is fixed in master now.

Thank you!  I just wanted to drop a note saying how much I've enjoyed using peewee and how appreciative I've been have your responsiveness to issues.
Issue: can't adapt type 'BackrefAccessor'
Author: aparcar
State: closed
Created: 2020-05-19 03:59:54

Hi, I'm trying to use `fn.array_agg` on a backref foreign key. However when trying to loop over the returned, model, I get the following error message:

```
Exception has occurred: ProgrammingError
can't adapt type 'BackrefAccessor'
```

My code looks something like that:

```python
    sources = target.sources.select(
        Sources.name,
        Sources.maintainer,
        fn.array_agg(Sources.results).alias("results"),
    ).group_by(Sources.name, Sources.maintainer)
```

Is it possible to *convert* the BackrefAccessor first to a dict or tuple to get this working? Or is there are different approach?

StackOverflow for questions.

You need to properly join on your relation and then array-agg over the result column you are trying to aggregate...
Issue: Peewee generates an SQL query with syntax errors
Author: theosotr
State: closed
Created: 2020-05-18 09:20:09

I have the following query

```python
mul = ((Listing.sale_price * Listing.yearly_rent)).cast('float').alias('mul')
avg = fn.avg(mul).alias('avg')
Listing.select(avg, mul).group_by(mul).where(mul > 100)
```
Peewee generates an SQL query with syntax error on `sqlite3`.

```
peewee.OperationalError: near "AS": syntax error
```

```sql
SELECT avg(CAST(("t1"."sale_price" * "t1"."yearly_rent") AS float) AS "mul") AS "avg", CAST(("t1"."sale_price" * "t1"."sale_price") AS float) AS "mul"
FROM "listing" AS "t1"
WHERE ("mul" > 100) GROUP BY "mul"
```

I would expect the following correct query. Peewee should ignore the alias of a compound column when used on an aggregate function

```sql
SELECT avg(CAST(("t1"."sale_price" * "t1"."yearly_rent") AS float)) AS "avg", CAST(("t1"."sale_price" * "t1"."sale_price") AS float) AS "mul"
FROM "listing" AS "t1"
WHERE ("mul" > 100) GROUP BY "mul"
```

Did you try this?

```python
mul = ((Listing.sale_price * Listing.yearly_rent)).cast('float')
avg = fn.avg(mul).alias('avg')
mul = mul.alias('mul')
```

Thank you; yes, this works!
But, I would expect peewee to implicitly turn off alias when used in irrelevant cases.

To do this it would have to recursively inspect every sub-node in the tree... eg. a function call would have to recursively inspect each of it's arguments. Peewee doesn't do this to avoid overhead. The workaround I suggested is the best way to handle it.
Issue: peewee.OperationalError: no such column: t2.subforum_id
Author: LuaFan2
State: closed
Created: 2020-05-17 13:21:54

I have such models. Everything looks good

```
class Cats(db.Model):
     name = CharField()

 class Forums(db.Model):
     cat = ForeignKeyField(Cats)
     name = CharField()
 
 class SubForums(db.Model):
     forum = ForeignKeyField(Forums)
     name = CharField()
     description = CharField(default="")
 
     def messages(self):
         print("Messages of " + str(self.id))
         return Messages.select().where(Messages.topic.subforum == self.id).count()
     def topics(self):
         print("Topics of " + str(self.id))
         return Topics.select().where(Topics.subforum == self.id).count()
     #def last(self):
         #last = Messages.select().where(Messages.topic.subforum == self.id).order_by(Messages.created.desc()).first()
 
         #return [last.topic.title, last.author.name, last.created.strftime("%m.%d.%Y %H:%M:%S")]
 
 class Topics(db.Model):
     author = ForeignKeyField(User)
     subforum = ForeignKeyField(SubForums)
 
     title = CharField()
     text = CharField()
 
 class Messages(db.Model):
     topic = ForeignKeyField(Topics)
     author = ForeignKeyField(User)
 
     text = CharField()
     created = DateTimeField(default=datetime.datetime.now)
```

But when I call messages() function I got 

`peewee.OperationalError: no such column: t2.subforum_id`

Why?Is this my fault or peewee?

I'm guessing you added that column to your model class without also ensuring it was added to the database. Try dropping and recreating your table, or migrating the schema using the migrate extension.

Please post questions on stackoverflow from now on. This is the issue tracker.