akumoth/peewee-issues huggingface dataset:
Issue: .get() Raises IndexError which Results in Noisy Stack Trace
Author: mkmoisen
State: closed
Created: 2022-03-02 18:58:11

Performing `.get()` for a row that does not exist will result in a verbose stack trace that starts with an IndexError.

I think it would look a bit cleaner if the IndexError did not print in the stack trace. If you agree, I would be happy to make the PR. Let me know.

```
Tweet.get(Tweet.id==0)
```

Results in:

```
Traceback (most recent call last):
  File "C:\test\lib\site-packages\peewee.py", line 6933, in get
    return clone.execute(database)[0]
  File "C:\test\lib\site-packages\peewee.py", line 4306, in __getitem__
    return self.row_cache[item]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\test\lib\site-packages\peewee.py", line 6485, in get
    return sq.get()
  File "C:\test\lib\site-packages\peewee.py", line 6936, in get
    raise self.model.DoesNotExist('%s instance matching query does '
__main__.TweetDoesNotExist: <Model: Tweet> instance matching query does not exist:
SQL: SELECT "t1"."id", "t1"."content", "t1"."timestamp", "t1"."user_id" FROM "tweet" AS "t1" WHERE ("t1"."id" = ?) LIMIT ? OFFSET ?
Params: [0, 1, 0]
```
-----


We could remove the IndexError from this stack trace by changing the following:

```
    def get(self, database=None):
        clone = self.paginate(1, 1)
        clone._cursor_wrapper = None
        try:
            return clone.execute(database)[0]
        except IndexError:
            sql, params = clone.sql()
            raise self.model.DoesNotExist('%s instance matching query does '
                                          'not exist:\nSQL: %s\nParams: %s' %
                                          (clone.model, sql, params))
```

to:

```
    def get(self, database=None):
        clone = self.paginate(1, 1)
        clone._cursor_wrapper = None
        result = clone.execute(database)
        if not result:
            sql, params = clone.sql()
            raise self.model.DoesNotExist('%s instance matching query does '
                                          'not exist:\nSQL: %s\nParams: %s' %
                                          (clone.model, sql, params))
        return result[0]

```

This change results in the following clean stack trace:

```
>>> Tweet.get(Tweet.id==0)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\test\lib\site-packages\peewee.py", line 6485, in get
    return sq.get()
  File "C:\test\lib\site-packages\peewee.py", line 6935, in get
    raise self.model.DoesNotExist('%s instance matching query does '
__main__.TweetDoesNotExist: <Model: Tweet> instance matching query does not exist:
SQL: SELECT "t1"."id", "t1"."content", "t1"."timestamp", "t1"."user_id" FROM "tweet" AS "t1" WHERE ("t1"."id" = ?) LIMIT ? OFFSET ?
Params: [0, 1, 0]
```


I'm going to pass for now.
Issue: My query should  return data  ,but  the result is sql; What should I do?
Author: wbzdssm
State: closed
Created: 2022-03-02 14:31:42

My query should  return data  ,but  the result is sql; What should I do?

sql just like:



    query =  FunctionName.select(
            *query_fileds
        ).where(
            *cond
        ).group_by(
            FunctionName.appid
        ).scalar(as_tuple=True)




Read the documentation. Follow the quickstart. Try iterating over your `query`.

Post questions on stackoverflow.

@wbzdssm When I use `scalar` it returns data.

What is `FunctionName`, is that a table?

Please add a reproducible example.

> FunctionName

yes , it' a table; 

@wbzdssm Can you add a reproducible example?

You can use the sqllite's in memory for convenience.

```


from peewee import *

db = SqliteDatabase(':memory:')

class BaseModel(Model):
    class Meta:
        database = db


class FunctionName(BaseModel):
    name = TextField()


db.create_tables([FunctionName])

function_name = FunctionName.create(name='foo')

query = FunctionName.select(
    fn.MAX(FunctionName.name)
).where(
    FunctionName.id == function_name.id
).group_by(
    FunctionName.id
).scalar(as_tuple=True)

print(query)
```

This will output `('foo',)` as expected.

tks
Issue: How to lazily declare a database connection?
Author: jorgecarleitao
State: closed
Created: 2022-03-02 12:52:10

The idiom

```python
from peewee import *

db = ...

class BaseModel(Model):
    class Meta:
        database = db
```

found in the guide, docs, etc. suggests that we have to either write the db credentials on the source code or read the credentials from environment on the root of the module (i.e. executed when we `import` the module).

However, database connections, and in particular secrets, are often only available after importing a module. Two examples:

* when the secrets are stored on a secret vault and we need to query said vault to retrieve the DB user/password, which we may want to read asynchronously, or every X days if the secret rotates.
* in CLI applications, where the secret may be inputted by the user as a CLI parameter or terminal input

Is there a "standard" or recommended idiom for these use cases? Ideally a way whereby we do not have to write something like

```python
def get_models(username, password):
    db = ... username, password

    class BaseModel(Model):
        class Meta:
            database = db
```

and then import each of the models into the module (or something).

More generally, is there a way to decouple database-specific properties from the creation of the ORM models?


If you had read the docs you would have found:

* http://docs.peewee-orm.com/en/latest/peewee/database.html#run-time-database-configuration
* http://docs.peewee-orm.com/en/latest/peewee/database.html#dynamically-defining-a-database
* http://docs.peewee-orm.com/en/latest/peewee/database.html#setting-the-database-at-run-time

Sorry for the noise.
Issue: add GitHub URL for PyPi
Author: andriyor
State: closed
Created: 2022-03-01 19:28:34

Warehouse now uses the project_urls provided to display links in the sidebar on [this screen](https://pypi.org/project/requests/), as well as including them in API responses to help the automation tool find the source code for Requests.

Docs: [packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/#project-urls](https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/#project-urls)
Issue: bulk_update for datetime fail
Author: LLjiahai
State: closed
Created: 2022-02-28 10:17:13


![aaa](https://user-images.githubusercontent.com/30220658/155965562-7511c03e-5de1-4df0-91b1-805303546976.png)




Im always surprised people can take the time to open the issue and then completely fail to explain their actual issue.

oh my god stupid thing to do,When I try to use "bulk_update" function update mysql_db datetime field, it doesn't take effect

It works fine for me:

```python
class Reg(Model):
    key = CharField()
    timestamp = DateTimeField()

# Create 3 objects w/timestamp of Mar 1, 2, and 3rd.
regs = [Reg.create(key='k%s' % i, timestamp=dt(2022, 3, i))
        for i in (1, 2, 3)]

# Change timestamps to be Apr 1, 2, and 3.
for reg in regs:
    reg.timestamp = reg.timestamp.replace(month=4)

# Perform update.
Reg.bulk_update(regs, fields=[Reg.key, Reg.timestamp])

# Verify timestamps updated correctly.
for reg in Reg.select().order_by(Reg.key):
    print(reg.key, reg.timestamp)
```

Output

```
k1 2022-04-01 00:00:00
k2 2022-04-02 00:00:00
k3 2022-04-03 00:00:00
```
Issue: Accept NoneType value as a default; Verify arguments while initializing the model object
Author: hung3a8
State: closed
Created: 2022-02-24 08:54:48

In [`peewee.Model.__init__`](https://github.com/coleifer/peewee/blob/528e014c640e0743c7a34069ca31754d79cc3de5/peewee.py#L6321), we set the values for the fields like this:
```python
        for k in kwargs:
            setattr(self, k, kwargs[k])
```
Let's say I am working on a `User` model with some fields like `username` or `password`, but somewhere in my code, I accidentally create an object like this:
```python
user = User(usernam='hung3a8', password='secret')
```
As you can see, I accidentally passed `usernam` instead of `username`. But as Peewee automatically set a new attribute for this arg instead of raising any kind of errors, which makes debugging becomes an actual hell. So instead of searching for the typo, maybe we should have a proper argument validation, or set those required fields as arguments and optional fields with default values as optional arguments (just like Django ORM and SQLAlchemy).

---

For some reason, `Field(default=None, null=True)` doesn't set the field value to `None`. In [`peewee.Metadata.add_field`](https://github.com/coleifer/peewee/blob/master/peewee.py#L6043), fields with `default=None` are passed, and their values are set to the pythonic default values (not sure how to call it) like `''` for CharField.
Even though we can set `NoneType` value for the field while initializing the object, doing that repetitively is troublesome. For example, with the same `User` model from the previous issue, now I also have a field called `email`, which is not required and nullable. While we can set it like this:
```python
user = User(username='hung3a8', password='secret', email=None)
```
Doing like this is much more cleaner:
```python
user = User(username='hung3a8', password='secret')  # We wont need to care about `email` anymore if the default value is set to None.
```

The first issue -- sorry this is not something I intend to change. Just write a wrapper or a classmethod constructor that does the validation if you want.

>In peewee.Metadata.add_field, fields with default=None are passed, and their values are set to the pythonic default values (not sure how to call it) like '' for CharField.

This doesn't sound right to me at all.

```python
class User(Model):
    username = CharField()
    password = CharField()
    email = CharField(null=True)
    class Meta:
        database = db

db.create_tables([User])

u = User(username='foo', password='secret')
print(u.username, u.password, u.email)
u.save()

u = User.get(User.username == 'foo')
print(u.username, u.password, u.email)
```

This correctly initializes email to None and stores it as None. It is also None when read back from the db:

```
$ python x.py 
foo secret None
foo secret None
```
Issue: Add and_, or_ functions ?
Author: mkmoisen
State: closed
Created: 2022-02-23 03:58:24

Would you consider adding a public `and_` and `or_` function to peewee?

SQLAlchemy has this which helps readability, compared to the bitwise operators.

```
User.select().join(
     Tweet, JOIN.OUTER_JOIN, on=(Tweet.user == User.id)
).where(
     User.id == 123,
     or_(
        Tweet.id.is_null(),
        Tweet.timestamp > datetime.now() - timedelta(hours=1)
    )
)
```

vs

```
User.select().join(
     Tweet, JOIN.OUTER_JOIN, on=(Tweet.user == User.id)
).where(
    User.id == 123,
    (
        (Tweet.id.is_null()) | (Tweet.timestamp > datetime.now() - timedelta(hours=1))
    )
)
```

The bitwise style can be especially hard to read when you have some complicated, multi nesting where clause with many ands and ors.

I noticed you already do this logic in in `where` and `orwhere` functions in `peewee.py`. Something like the following should suffice:

```
def or_(*expressions):
    return reduce(operator.or_, expressions)

def and_(*expressions):
    return reduce(operator.and_, expressions)
```


Peewee doesn't need these - you can just `from operator import or_` and use that directly.
Issue: Add better support for left_outer_join convenience function
Author: mkmoisen
State: closed
Created: 2022-02-22 18:56:37

The `Source` class has a `left_outer_join` convenience function which calls `join`. However, unlike `join`, the `left_outer_join` function is not overridden in `Select` and `ModelSelect`.

Currently, this means that doing `User.select().left_outer_join(Tweet)` returns a `Join` object, not a `ModelSelect`. 

This is inconsistent with `User.select().join(Tweet)` which returns a `ModelSelect`. For example the following fails today because there is no `switch` function defined on the `Join` object:

    Tweet.select().left_outer_join(User).switch

The `Join` object also has no `where`, `sql`, or `execute` functions, which effectively means this `left_outer_join` function cannot be used today.

---

This PR adds the `left_outer_join` convenience function to the `Select` and `ModelSelect` classes.

---

One thing to consider is whether or not it might be better to name this `outerjoin` instead of `left_outer_join`. `outerjoin` would be [consistent with SQLAlchemy](https://docs.sqlalchemy.org/en/14/core/selectable.html#sqlalchemy.sql.expression.FromClause.outerjoin) if that was desirable.  Or perhaps just shorten `left_outer_join` to `outer_join` or `left_join`.

If you wanted to, I think it *might* be safe to rename this function without breaking backwards compatibility. Because the current Tweet.select().left_outer_join(User) is returning a `Join` object which cannot be used in any context at least that I can think of. E.g., it has no execute(), where(), sql() functions.

----

I added a test for this which is passing. However I found that a few other tests are failing, my guess is that they were already failing before this change. Thee following failed for me: test_close_stale (tests.pool.TestPooledDatabase), test_timestamp_field_from_ts (tests.fields.TestTimestampField), test_timestamp_field_parts (tests.fields.TestTimestampField)
. 


Thanks, looks good!
Issue: How contribute to project ?
Author: ParfaitD9
State: closed
Created: 2022-02-22 14:11:34

I just want to know how to contribute to the project. I'm very interested. 


#peewee on libera.chat (IRC) is probably the best place, but it's fairly quiet and mostly used for helping answer questions or troubleshooting issues. To contribute you just need the following:

* have an idea for how to improve the library
* implement it (optionally checking here to see if the improvement is wanted/desirable)
* notify maintainers about improvements

Thanks for the reply.

Le mar. 22 févr. 2022 à 3:20 PM, Charles Leifer ***@***.***>
a écrit :

> Closed #2531 <https://github.com/coleifer/peewee/issues/2531>.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/coleifer/peewee/issues/2531#event-6118428127>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ATBUOTIBGYLOM2BRCEMKEFLU4OLSZANCNFSM5PBQJOAQ>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>
Issue: Memory Caching Question
Author: mkmoisen
State: closed
Created: 2022-02-21 17:20:50

Not a bug, just a question.

In the documentation it states the following:

> Subsequent iterations of the same query will not hit the database as the results are cached. To disable this behavior (to reduce memory usage), call Select.iterator() when iterating.

I'm not clear in what circumstances I would want to call `iterator()` to reduce memory consumption.

In my web app I am commonly doing something like the following:

```
@app.get('/tweets/')
def get_tweets():
    tweets = list(Tweet.select())  # as opposed to list(Tweet.select().iterator())
    return TweetSchema.dump(tweets, many=True)
```

Am I using twice as much memory as required if I do `tweets = list(Tweet.select())` compared to `tweets = list(Tweet.select().iterator())` ?

I don't think so, because of the following shows that it is by reference, not by value:

```
    sel = Tweet.select()
    tweets = list(sel)

     assert sel[0] is tweets[0]
     assert id(sel[0]) == id(tweets[0])
```



So peewee select queries are not executed until iterated over (although explicitly calling `execute()` on a Select query *will* execute it, returning a cursor wrapper, but that's kinda beside the point).

However, this code *only* issues and executes a single query:

```python
query = Tweet.select().limit(50)

# Query is executed here.
for tweet in query:
    ...

# These iterations and other operations do not re-execute the query.
for tweet in query:
    ...

tweets = list(query)

first_tweet = query[0]
...
```

During the first iteration the resulting rows are cached, in memory, so that subsequent iterations and even indexing will not hit the database again. This is what is meant by caching in Peewee -- rows are cached so that the query result can be operated on more than just a single-shot iteration.

If, however, you are processing a ton of rows iteratively and all you need is a single iteration, you can save memory by using `.iterator()`. When using `.iterator()`, attempting to iterate over the query a second time will yield no rows/no data.

I'm not sure why you are coercing it to a list - that should be unnecessary in any case, since simply iterating and re-iterating the query will not yield additional executions.

I'm just coercing to a list for demonstration purposes. In terms of forcing the select to execute, calling `list(Tweet.select())` is the same as `for tweet in Tweet.select():`.

If I need to load all the rows into memory anyways to accomplish some task, is there any benefit to calling `iterator()`? I'm pretty sure the answer is no, right?

For example, there is no benefit to calling `iterator()` in this this REST API example:

```
@app.get('/tweets/')
def get_tweets():
    tweets = Tweet.select().iterator()
    return TweetSchema.dump(tweets, many=True)

```

Is that right?


No there's no reason to worry about it unless, like the docs say, you intend to iterate over so many rows that memory usage would become an issue.
Issue: Default rule is not being applied in SQL
Author: AnitaErnszt
State: closed
Created: 2022-02-09 10:04:11

When creating new table the default constraints are not applied in SQL.

Example:
```
class Test(BaseModel):
    not_nullable = CharField()
    unique = CharField(null=True, unique=True)
    default_true = BooleanField(default=True)
    default_0 = IntegerField(default=0)
```

If I run print_table_sql(Test) I get below, which is seems to be the SQL statement executed.
```
class Test(BaseModel):
    not_nullable = CharField()
    unique = CharField(null=True, unique=True)
    default_true = BooleanField(default=True)
    default_0 = IntegerField(default=0)
```
    
I need to make sure these default rules are also applied in SQL. 
Is there something I'm missing? If there's no straightforward way of doing this what's the best way to get the default values out of the models?

Edit:
Managed to get the defaults using below code. Is there a better way to achieve the same?
```
for model in models:
    for name, field in model._meta.__dict__["fields"].items():
        print(field.default)
```

>I need to make sure these default rules are also applied in SQL.

That's done by adding an explicit constraint:

```python
default_0 = IntegerField(constraints=[SQL('default 0')])
```

Please consult the docs. I don't write them for fun.

http://docs.peewee-orm.com/en/latest/peewee/models.html#default-field-values
Issue: Transaction not working with migrate
Author: BobBDE
State: closed
Created: 2022-02-04 13:16:52

Hello, 

I have a problem with the peewee migration. I can't make transactions work with the migrate. It is always committed, I can't rollback. 
It should be possible as the documentation says in the warning section here: https://docs.peewee-orm.com/en/latest/peewee/playhouse.html#example-usage

Here is an exemple code where the rollback does nothing and the alter table is commit. 
```migrator = MySQLMigrator(db)
   with db.atomic() as txn:
      migrate(migrator.add_column("table_name", 'new_column', CharField(default='')))
      # this should rollback the migrate
      txn.rollback()
```
When I reach the ```txn.rollback()``` the action is already committed and I can see the new column in my table. The rollback does nothing. 

Is there something I do wrong ? I do use transaction in other senario like basic CRUD operation and thoses works as expected. 

I'm using peewee v3.14.8

Are you using MySQL? It's quite possible your database does not support transaction DDL.

Transactions are working fine with the migrator generally, I assume this is down to your database not supporting it:

```python
class Reg(Model):
    key = TextField()
    class Meta:
        database = db

db.drop_tables([Reg])
db.create_tables([Reg])

migrator = SqliteMigrator(db)
with db.atomic() as txn:
    migrate(migrator.add_column('reg', 'new_col', TextField(default='')))

    print([x.name for x in db.get_columns('reg')])
    txn.rollback()

print([x.name for x in db.get_columns('reg')])
db.drop_tables([Reg])
```

This correctly prints:

```
['id', 'key', 'new_col']
['id', 'key']
```

Indeed, mysql doesn't support rollback for DDL, I didn't know that. 

It wasn't a peewee problem, sorry for the bothering

Thank you for your answer :)
Issue: Temporary Database in SQLite
Author: RAbraham
State: closed
Created: 2022-02-03 11:15:15

Hi,

How do I create a temporary SQLite database[1] using `peewee`?
In Python, I would normally create it like this with an empty string
```python
import sqlite3
db = sqlite3.connect("")
```

[1] - Section 'Temporary Databases' at https://www.sqlite.org/inmemorydb.html

Peewee has excellent documentation, here:

http://docs.peewee-orm.com/en/latest/index.html

I suggest searching that for the string ':memory:', for example. 

`SqliteDatabase(':memory:')`

Hi,
Thanks for your quick response.
I think what I want may not be satisfied by the ':memory' config value. I'd like a temporary table which is slightly different and a different option than an `in memory` database which I think will be created by `SqliteDatabase(':memory:')`

From the above sqlite documentation:

> The sole difference is that a ":memory:" database must remain in memory at all times whereas parts of a temporary database might be flushed to disk if the database becomes large or if SQLite comes under memory pressure.

Therefore, there are different options in sqlite
- For a pure in-memory database, it's ` sqlite3.connect(":memory:")` which is what `SqliteDatabase(':memory:')` is a wrapper for?
- For a temporary database, it's `sqlite3.connect("")`, unless `SqliteDatabase(':memory:')` creates a temporary table and not an in memory table? I did read the documentation a bit  before I posted. so if I have missed this part, I apologize.

I was not aware of this usage. In that case I'd recommend just using Python's tempfile facilities to initialize it with a named temp file, or explicitly mark the database as not being deferred:

```python
db = SqliteDatabase('')
db.deferred = False
```

I'll try that. Thank you very much.
Issue: Added optional use_column_names instead of field names in model_to_dict
Author: deduble
State: closed
Created: 2022-01-30 07:52:37

I am using this in model_to_dict in jobs because it makes the created python object keys consistent with the database column names when communicating through web apps. Despite column names are not case sensitive when querying, language mediums are, so it goes hand in hand with pwiz db_model.py tool. I didn't know what to do with backrefs though, hence left those lines. 

I'll pass on this, thanks.