akumoth/peewee-issues huggingface dataset:
Issue: Found some potential code quality issues in peewee
Author: pnijhara
State: closed
Created: 2020-08-23 21:52:42

I was reading peewee's documentation last month and so I thought of forking this repo and running a code quality analysis on it.
In that analysis, [DeepSource](https://deepsource.io/) found somewhat around 1000 code quality issues.

Some interesting issues found in the source code:
- [(Bug-risk) Multiple imports for an import name detected](https://deepsource.io/gh/pnijhara/peewee/issue/PYL-W0404/)
- [(Bug-risk) `hasattr` used to check if the object is callable](https://deepsource.io/gh/pnijhara/peewee/issue/PTC-W0035/)
- [(Performance) Built-in function `len` used as a condition](https://deepsource.io/gh/pnijhara/peewee/issue/PYL-C1801/)
- [(Security) Assert statement used outside of tests](https://deepsource.io/gh/pnijhara/peewee/issue/BAN-B101/)

What's so good?
All the above-stated issues can be automatically fixed by autofix tool of DeepSource. Look at this autofix session &rarr; https://deepsource.io/gh/pnijhara/peewee/autofix/54d8d678-156a-4aa1-aea3-9fb63ebe4f89/ and the [PR](https://github.com/pnijhara/peewee/pull/3)

> Code that conforms stylistically with the rest of the Peewee codebase.

Quoting from peewee's documentation

Since there is no coding style followed in the project, I decided to run a Transformer as well. Since the last release, DeepSource can automatically do that and create a PR on the project.

-  [ Automated transformation](https://deepsource.io/gh/pnijhara/peewee/transform/34b7ba8d-23af-456f-bcd7-b36f7cd97713/) using Black, and [the subsequent PR](https://github.com/pnijhara/peewee/pull/1).

Please let me know what issues you'd like to focus on fixing and I'd be happy to take a look into it. Also, you can choose to hide certain types of issues too (if you wish to ignore them or you believe it is a false positive). I'd also be happy to send a patch with the DeepSource configuration file with the required fixes.

You can find the required configuration file for DeepSource [here](https://github.com/pnijhara/peewee/blob/pnijhara-patch-1/.deepsource.toml).
Issue: Lost type of aliased column from subquery
Author: Nurbel
State: closed
Created: 2020-08-14 14:52:40

Hi,

when using aliased colums in a subquery, peewee seems to loose the type of the column

In the following script, I access a date from a subquery, with and without an alias.
Without the alias, I get a python date as expected, but with the alias, I get a string

```python
from peewee import *
from datetime import date
DB = SqliteDatabase('/tmp/test_subquery.db')


class MyModel(Model):
    date = DateField()

    class Meta:
        database = DB


DB.create_tables([MyModel])
MyModel.replace(date=date(2020, 1, 1)).execute()

mod2 = MyModel.alias()
sub = mod2.select(mod2.date, mod2.date.alias('aliased'))
q = (MyModel.select(MyModel.date, sub.c.date, sub.c.aliased)
     .join(sub, on=(sub.c.aliased == MyModel.date))
     )

print(q.dicts().get())
```
> {'date': datetime.date(2020, 1, 1), 'aliased': '2020-01-01'}

Am I doing something wrong? From the doc, using aliases seemed to be the recommended way to access the subquery columns


That's correct, and peewee will figure it out correctly most of the time, but when you're pulling stuff up from subqueries you may need to explicitly tell peewee how to coerce it.

Did you try:

```python

.select(MyModel.date, sub.c.date, sub.c.date, sub.c.aliased.coerce(MyModel.date.db_value))
```

Your proposition does not seem to work.

the best thing I managed to do was to use MyModel.date.python_value on the result
```python
MyModel.date.python_value(row['aliased'])
```

Whoops, I meant `.coerce(MyModel.date.python_value)` -- did you try that?

Yes,I tried it did not work either

Sorry, I had my APIs mixed-up. I've added a new API for this case, which *may* change somewhat before the next release -- but this should provide a way forward for these cases that aren't covered by the existing heuristics.

The new API would be:

```python

.select(..., subq.c.aliased.converter(callable))
```

And `callable()` would be called on the corresponding row value taken from the cursor.

Now I don`t use subquery .  But after used cast sting to int,I use alias I get string not a int value

Possibly you are aliasing it to the same name as used by an existing char/text field on your model?

> Possibly you are aliasing it to the same name as used by an existing char/text field on your model?

You are right, now i use round function to cast string to decimal

If you alias an expression to the name of an existing column, peewee assumes you want to use that column's conversions for the data. So you might try also just using a different name - one that doesn't coincide with an existing field.
Issue: Misleading query generated on SQLite
Author: theosotr
State: closed
Created: 2020-08-14 13:39:01

I run the following query on SQLite.

```python
q1 = Table1.select(Table1.colA).objects()
q2  = Table2.select(Table2.colB).order_by(Table2.colB).limit(1).objects()

q1.union(q2)
```

the query generated by `peewee` is misleading, and not equivalent to the initial one

```sql
SELECT "t1"."colA" FROM "table1" AS "t1" UNION SELECT "t2"."colB" FROM "table2" AS "t2" ORDER BY "t2"."colB" LIMIT 1 
```

In particular, the `ORDER BY` and `LIMIT` clauses are applied to the combined query produced by the `UNION` clause, and *not* to the second sub-query as intended. 

Sqlite does not support parentheses on union queries, nor does it support ORDER BY or LIMIT on the individual queries that comprise the union. Read the sqlite docs:

https://sqlite.org/lang_select.html#compound_select_statements

>As the components of a compound SELECT must be simple SELECT statements, they may not contain ORDER BY or LIMIT clauses.

Thanks
Yes, I figured out what happened here. However, peewee's result was misleading because someone would expect peewee to produce a query that would fail on SQLite (since ORDER_BY and LIMIT are not supported on individual queries).
Issue: doc website seems to be hacked
Author: azs0309
State: closed
Created: 2020-08-11 17:31:31

The document website http://docs.peewee-orm.com/en/3.6.0/peewee/quickstart.html seems to be hacked and whenever I click a link pointing me to fryruejoust.cam

It seems to be working fine for me. This is the output of `curl http://docs.peewee-orm.com/en/3.6.0/peewee/quickstart.html`:

https://gist.github.com/coleifer/d6450e6d5bdcb4460cd6bbf36b8dcb4c
Issue: left_outer_join doesn't works as expected
Author: slavkoja
State: closed
Created: 2020-08-11 17:03:20

I do not know, if i am doing something wrong, but query with `left_outer_join()` doesn't work for me. I can reproduce it with this simple example:
```python
import peewee as pw

DBFILE = ":memory:"
db = pw.SqliteDatabase(DBFILE, pragmas={'foreign_keys': 1,})

class BaseModel(pw.Model):
    class Meta:
        database = db

class Domains(BaseModel):
    id = pw.PrimaryKeyField(null=True)
    dname = pw.TextField()

class DAliases(BaseModel):
    dalias = pw.TextField()
    domain = pw.ForeignKeyField(Domains)
```
Simple `join()` works without problems:
```python
q = (Domains
     .select()
     .join(DAliases, pw.JOIN.LEFT_OUTER)
     .where(Domains.id==1)
    )
```
But when i replace `join()` with `left_outer_join()`:
```python
oq = (Domains
     .select()
     .left_outer_join(DAliases)
     .where(Domains.id==1)
    )
```
It throws exception:
```python
Traceback (most recent call last):
  File "pwerr.py", line 36, in <module>
    oq = (Domains
AttributeError: 'Join' object has no attribute 'where'
```
I am surprised, because in source i see, that `left_outer_join()` returns the same object as `join()`, the difference is only in predefined `join_type`... Am i doing something wrong or problem is elsewhere?

The `left_outer_join()` method is implemented on a lower-level of the query-builder and does not take into account important aspects of the model layer.

You should stick to using the standard ways of doing things, namely `.join()` when using models.
Issue: Fatal Python error: GC object already tracked
Author: simpassi
State: closed
Created: 2020-08-11 16:27:47

I'm having a weird intermittent error when all of the following conditions are met:
* Using an in-memory sqlite3 database initialized with thread_safe=False, check_same_thread=False
* Running in a different thread than the one that initialized the database
* Accessing an object's _meta.backrefs values

Running inside the debugger in vs.code will make this occur much more frequently.

The actual Python error is:
```
Fatal Python error: GC object already tracked
```

When the crash happens, all the other threads are in wait() and the crashed one is here:
```
  File "/opt/conda3/lib/python3.7/site-packages/peewee.py", line 4395 in __get__
```

I realize these conditions are somewhat obscure - I'm talking through a REST API to a backend that is running with the same data model but with a Postgres database, and using the same model on the client end and updating the responses into the local sqlite3 in-memory database. The data travels over using *playhouse.shortcuts.model_to_dict* and reversed using dict_to_model. Just wondering if this is something easily resolved or if I need to bring all database queries back to the main thread...

Peewee version: 3.13.2

Python version: 3.7.6 (Anaconda, MacOS)

I'd take it up with the cpython / pysqlite folks. Sharing sqlite connections across python threads is weird, and peewee uses a connection-per-thread model anyways so I am not even sure how that is working for you.

You probably need a minimal repro using threads and stdlib sqlite3.
Issue: After upgrading from version 2.x to 3.x getting, TypeError: _connect() missing 1 ...
Author: wilberh
State: closed
Created: 2020-08-09 20:07:58

After upgrading from version 2.x to 3.x getting, I'm getting the error below.  It worked fine in version 2.x  


```
  File "/.../lib/python3.6/site-packages/peewee.py", line 403, in inner
    with self:
  File "/.../lib/python3.6/site-packages/peewee.py", line 2936, in __enter__
    self.db.connect()
  File "/.../lib/python3.6/site-packages/peewee.py", line 3035, in connect
    self._state.set_connection(self._connect())
TypeError: _connect() missing 1 required positional argument: 'database
```



http://docs.peewee-orm.com/en/latest/peewee/database.html?highlight=Adding%20a%20new%20Database%20Driver#adding-a-new-database-driver

```
from peewee import Database
import foodb  # Our fictional DB-API 2.0 driver.


class FooDatabase(Database):
    def _connect(self, database, **kwargs):
        return foodb.connect(database, **kwargs)

def get_database_connection():
    return FooDatabase(database, **db_config)
```

`_connect()` does not take a database parameter in 3.x. It is `self.database`.
Issue: SQLite database generates GLOB for %, however GLOB is not LIKE…
Author: zx80
State: closed
Created: 2020-08-08 16:29:33

The documentation says that `%` stands for `LIKE` in query expressions, but the traces shows a `GLOB` instead with SQLite, hoewever the regexp syntax is not the same between the two operators, eg `LIKE 'h%'` is `GLOB 'h*'`.

Version affected: *3.13.3*

Code:

```Python
    res = Store.select().where(Store.key % PARAMS['filter']).order_by(Store.key)
```

Trace:

    INFO:root:db = <peewee.SqliteDatabase object at 0x7ffac44c0c10>
    DEBUG:peewee:('SELECT "t1"."key", "t1"."val" FROM "store" AS "t1" WHERE ("t1"."key" GLOB ?) ORDER BY "t1"."key"', ['%h'])

In the [source](https://github.com/coleifer/peewee/blob/master/peewee.py) on line 3347:

```Python
    operations = { 'LIKE': 'GLOB', 'ILIKE': 'LIKE' }
```

Indeed, SQLite `LIKE` is case insensitive, however `GLOB` is not `LIKE` at all. ISTM that it would be more helpful to generate a not implemented exception than switching to another operator which breaks the portability. 

This has been this way for many years and I've no plans to change it. I do understand that it may have been confusing.

Hmmm… Indeed it is pretty confusing.

![s1597080267 5538623](https://user-images.githubusercontent.com/119974/89811582-7fe09c00-db04-11ea-9d0f-d01e1900180d.png)
Issue: Using bitwise (&|~)in a query
Author: TaceyWong
State: closed
Created: 2020-08-06 07:09:00

IN http://docs.peewee-orm.com/en/latest/peewee/query_operators.html

.bin_and(value) -> Binary AND

.bin_or(value) ->  Binary OR

but,I can not find the replace of Binary NOT .How can I write a peewee-style c=a&(~b)?



Since it's a unary operator, you can probably get it working with something like:

```python

def bin_not(value):
    return NodeList((SQL('~'), value))
```
Issue: CTE executed multiple times
Author: frafra
State: closed
Created: 2020-08-03 15:34:32

Hi,
Peewee defines CTE in subqueries, even if that means to compute the same thing twice.

For example: `SELECT ... FROM ... JOIN (WITH cte ... SELECT ... FROM cte) JOIN (WITH cte ... SELECT ... FROM cte) ...`

I have a rather complex example I can show, but it surprises me that the CTE is not placed at the beginning, as that has been defined once.

Please see the docs for cte usage. I get the sense you're missing something:

http://docs.peewee-orm.com/en/latest/peewee/querying.html#common-table-expressions

The tests also have numerous examples.

My misunderstanding was based on the assumption that `.with_cte(...)` should have been added to each [sub]query which refers to the CTE, while it just adds a `WITH ... AS` at the beginning of the [sub]query.

My problem has been fixed by using `.with_cte(...)` just once, at the very end of the main query.
Issue: Add the auto_now parameter to the TimestampField field
Author: ghost
State: closed
Created: 2020-08-02 07:08:14

I want to add this parameter, when set to True, the database entry will be updated and changed to the current time

You can already do this by overloading the default datetime class, something like:
``` python
from datetime import datetime

import peewee as pw


# pylint: disable=locally-disabled, invalid-name
def NowDateTimeField(**kwargs):
    """ A peewee DateTimeField which defaults to 'now'
    """
    constraints = kwargs.pop('constraints', [])
    if not kwargs.get('null'): # also default to 'now' on insert
        constraints.append(pw.SQL('DEFAULT CURRENT_TIMESTAMP'))

    return pw.DateTimeField(
        default=datetime.now,
        constraints=constraints,
        **kwargs,
    )

```

There are a couple options:

* Specify default=datetime.datetime.now (no parens!!)
* Override the save() method
* Use a constraint
* Create a trigger to set new value on update
Issue: What does the `gametight` tag mean?
Author: pxeger
State: closed
Created: 2020-07-30 13:59:47

Random question - the tag `gametight` is used on only two other repos - kt and ukt, both by you (coleifer). What does it mean?

https://www.youtube.com/watch?v=McnTeuKtmfg&t=1m24s
Issue: Add ping reconnect mysql connection
Author: bagel
State: closed
Created: 2020-07-30 13:24:08

When mysql server restart or lost connection, connection not reconnect and database.is_closed is False.

Add ping(reconnect=True) auto reconnect.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 1987, in __len__
    self._ensure_execution()
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 1969, in _ensure_execution
    self.execute()
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 1886, in inner
    return method(self, database, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 1957, in execute
    return self._execute(database)
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 2129, in _execute
    cursor = database.execute(self)
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 3112, in execute
    return self.execute_sql(sql, params, commit=commit)
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 3106, in execute_sql
    self.commit()
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 2873, in __exit__
    reraise(new_type, new_type(exc_value, *exc_args), traceback)
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 183, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.8/dist-packages/peewee.py", line 3099, in execute_sql
    cursor.execute(sql, params or ())
  File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 170, in execute
    result = self._query(query)
  File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 328, in _query
    conn.query(q)
  File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 516, in query
    self._execute_command(COMMAND.COM_QUERY, sql)
  File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 750, in _execute_command
    raise err.InterfaceError("(0, '')")
peewee.InterfaceError: (0, '')

>>> database.is_closed()
False


See: https://github.com/coleifer/peewee/blob/master/playhouse/shortcuts.py#L174-L229

I use like this before, Why not use ping auto reconnect?
Issue: Inconsistency in treatment of '\' character
Author: theosotr
State: closed
Created: 2020-07-29 14:49:14

I have the following database schema

```sql
CREATE TABLE "table" (
"id" integer,
"str" varchar(50),
 PRIMARY KEY (id)
);
INSERT INTO "table"("id","str") VALUES (1,'bar_foo\baz');
INSERT INTO "table"("id","str") VALUES (2,'bar');
INSERT INTO "table"("id","str") VALUES (3,'bar\foo');
```

and the following query

```python
ret = Table.select(Table.id).where(Table.str.contains('\\'))
for r in ret:
    print(r.id)
```

This prints the following as expected

```
1
3
```

However, when I have this query

```python
ret = Table.select(Table.id).where(Table.str.contains('_foo\\'))
for r in ret:
    print(r.id)
```

I expect peewee to dump

```
1
```
but, peewee dumps nothing.

Depending on your database, you may need to double-up the backslash, as many databases treat it as the default escape.

Just use four: '\\\\\\\\'.

I think this behaviour needs to be hidden from the programmer. Besides, an ORM is abstraction layer over databases.
Because adding `\\\\` in the first query will make peewee to return nothing.

So when mixing `\` with '_', I suggest peewee add the following in its logic.
```python
str.replace('\\', '\\\\')
```

What database are you using?

Also: are you running the latest version from master branch?

I am using sqlite, and I am using the mater version of peewee.

>I am using the mater version of peewee.

I think I know what the issue is, in that case. I made a recent change to the "contains()" method so that "_" and "%" within the search-string are properly escaped -- as you may know, these are wildcards that have special meaning for partial-string search. So if a string contains "_" or "%", the expression is converted somewhat. I believe I can fix this with a small addition. Thanks for reporting.

Thanks!
Issue: Support queries of all types of pkey in `get()`
Author: wooparadog
State: closed
Created: 2020-07-29 08:51:23

In https://github.com/coleifer/peewee/issues/1835 get by primary key is added for `int`, but there're cases when primary key is other than integer, especially in analytics domain.

Using `Node` to exclude `Expression` seems a good idea, need more insight though.

I'm going to pass, but thanks.
Issue: Unable to pass database parameter to mysql-connector-python through peewee
Author: dshefman1
State: closed
Created: 2020-07-28 22:23:54

The database parameter 'tls_versions' does not appear to be getting passed through to mysql-connector-python. The [documentation ](http://docs.peewee-orm.com/en/latest/peewee/database.html#initializing-a-database) says, "Subsequent keyword arguments are passed to the underlying database driver when establishing the connection, allowing you to pass vendor-specific parameters easily". This does not appear to be the behavior I am witnessing when I pass the "tls_versions" parameter through peewee to mysql-connector-python.

When I connect directly with mysql-connector-python using the code below, I am able to connect successfully:

```
import mysql.connector

config = {'database': 'dbname',
 'user': 'username_so',
 'password': 'psswrd',
 'host': 'maria####-##-###-##.###.####.com',
 'port': 3307,
 'tls_versions': ['TLSv1.1', 'TLSv1.2']}

cnx = mysql.connector.connect(**config)

print(cnx.is_connected())
cnx.close()
```
Output:
> True

However, when I pass in the same configuration to peewee.MySQLDatabase() then I get an error message:

```
db = MySQLDatabase(**config)

db.get_tables() # This function and any other that connects to the db returns the same error message specified below
```
Error Message:

> ImproperlyConfigured: MySQL driver not installed!

I am pretty sure that the problem is with specifying the tls versions in peewee because I was getting the same error message with mysql.connector before I added in the additional 'tls_versions' parameter. Also, I get the same error message with peewee if I do not include the 'tls_versions' parameter at all.

My Setup:
Linux
Python 3.7
peewee==3.13.3
mysql-connector-python==8.0.21

You need to use `playhouse.mysql_ext.MySQLConnectorDatabase` to connect using the mysql-connector driver. That will resolve your issue.
Issue: passing custom/additonal metadata
Author: wilberh
State: closed
Created: 2020-07-24 19:23:35

Is there any way to pass additional metada when creating a table (in class Meta or some other way)?  
Meaning, adding additional information after table creation section - see bold section below.

Example: 

CREATE TABLE testing (
name VARCHAR
)
**COMMENT 'This is an internal table' 
ROW FORMAT DELIMITED 
NULL DEFINED AS '' 
STORED AS AVRO** 



No this is not supported
Issue: How to handle 'Duplicate entry' warnings when using `on_conflict_ignore()`?
Author: SilentGene
State: closed
Created: 2020-07-23 18:22:26

Hello, when using `insert.on_conflict_ignore().execute()` method to insert new entries, there are always warnings shown as below:
```bash
C:\Users\...\AppData\Local\Programs\Python\Python37\lib\site-packages\pymysql\cursors.py:170: Warning: (1062, "Duplicate entry 'http://link.aps.org/doi/10.1103/RevModPhys.90.015006' for key 'IX_link'")
  result = self._query(query)
```
Could I know how to hide or filter out these warnings?

The warning is not raised by peewee. The warning is not generated by pymysql. The warning is coming directly from the sql database.

For general information about warnings, check here:

https://docs.python.org/3/library/warnings.html#module-warnings

Next time please spend more time looking into it before just immediately opening an issue.

Thank you @coleifer . I have been struggling for this issue for some time, but didn't know this is coming from SQL database with my little experience in SQL.  I thought this originated from pymysql or peewee, so I've spent much time looking for solutions there.

With your suggestion, now I have successfully used the following code to suppress the warnings:
```python
import warnings

def save_entries(d):
    with warnings.catch_warnings():
        warnings.filterwarnings('ignore', '.*Duplicate entry.*')
 
        with mysql_db.atomic():
                PaperInfo.insert_many(d).on_conflict_ignore().execute()
```

Thanks again for your time.