akumoth/peewee-issues huggingface dataset:
Issue: Malformed column name returned from select() on modified field.
Author: davidbernat
State: closed
Created: 2019-07-26 18:58:22

```
class PointField(pw.Field):
    """Geometry field type Point, conforming to MySQL 8.0+, format is WKT: 'POINT(18 23)'"""

    # Field type in MySQL is Point with a capital P.
    field_type = 'Point'

    def python_value(self, value):
        return value

    def db_value(self, ewkt):
        """Converts EWKT format to MySQL database requirements, e.g., SRID=3857;POINT(10,10)"""

        # Does the ewkt start with "SRID=XXXX;...".
        # If so, remove the value from the geometry and parse/extract the srid
        s = ewkt.split(";")
        if s[0].startswith("SRID="):
            geo = ";".join(s[1:])
            srid = s[0].split("=")[1]
            return pw.fn.ST_GeomFromText(geo, srid)
        else:
            return pw.fn.ST_GeomFromText(ewkt)
```

```
class PadsLocations(AetherDataModel):
    name = pw.CharField(max_length=255, index=True)
    latlng = PointField()
    operator = pw.BooleanField()
    operator_name = pw.CharField(max_length=255, index=True)
```

```
s = PadsLocations.select(PadsLocations.name, pw.fn.ST_AsText(PadsLocations.latlng))
for r in adf.read(s):
    print(r.name, r.latlng)
```

`r.latlng` returns None. This is because the correct value returns with a key of "\`latlng)`".



`pw.fn.ST_AsText(PadsLocations.latlng).alias('latlng')` does return the appropriate result, as a monkey-patch.

Peewee relies on the driver to implement the db-api 2.0 spec. Part of this spec is that a cursor, after executing a query, contains a `description` property which tells us about the names of the columns being selected. This is determined by the database and then communicated to the driver. Presumably if you execute such a query in the mysql shell, the result column of such a function will not be "latlng" but something else.

So the correct thing to do, if you want to alias a function call to a given, is to explicitly specify the alias as you have done. Not a bug.

Consider the case where you're grabbing a COUNT() or something. You'd have `fn.COUNT(MyModel.id)` or whatever. You wouldn't want that to default to being the value of "MyModel.id" ... so you explicitly alias it: `fn.COUNT(MyModel.id).alias('count')`.

> as a monkey-patch

look up the definition of monkey-patch, you're using it incorrectly
Issue: SchemaMigrator.add_column problem
Author: richey-v
State: closed
Created: 2019-07-24 20:17:14

(using peewee 3.9.6 with sqlite3)

I am assuming the problem that I am reporting may be difficult to fix, so this Issue might be best resolved by adding a note to the relevant [documentation](http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#SchemaMigrator.add_column).

I needed a new Field in a Model and I naively added the new Field _in between_ two other Fields.  I wanted to provide a migration for the new Field on this Model, so I used `add_column`.  The new Field was *inserted* in between the other two Fields, but the column for the data was added at the *end* of each row for the Model.

Ideally, `add_column` gets fixed to put the new data column in the correct position, but if that is too difficult or takes a long time to be fixed, _please_ add a note recommending that new Fields be added after the last old Field.

What difference does it make what order the column is in the underlying storage? Absolutely none, truthfully.
Issue: Avoid situations where connections have same ts.
Author: itsmeczaharia
State: closed
Created: 2019-07-24 11:52:23

Hello,
I was able to reproduce the bug from this issue: https://github.com/coleifer/peewee/issues/1883
Adding a small random number to ts, avoids that race condition.

What's odd is that `time.time()` has microsecond precision on my machine. Granted, calling `time.time()` appears to take only ~80ns, but still there's a lot of stuff going on that would seem to make identical timestamps an impossibility. I'm fine with merging, but I can't wrap my head around how this could happen in the first place.

Fixed and merged an equivalent (but not broken) patch.
Issue: Unecessary index on table creation
Author: lucianobarcaro
State: closed
Created: 2019-07-23 19:31:51

I'm new to peewee (2 weeks now) and I noted an unecessary index on table creation.

Simple case:
 
```
class Usuario(BaseModel):
    nome = CharField()

 
class UsuarioAcesso(BaseModel):
  usuario = ForeignKeyField(Usuario)
  dataAcesso = DateTimeField()
  fromIP = CharField()
  class Meta:
      primary_key = CompositeKey('usuario', 'dataAcesso')
```

On table UsuarioAcesso, peewee create 2 indexes:
Primary key as expected (usuario_id, dataAcesso)
and
usuario_acesso_usuario_id (usuario_id)

This second index is already covered with primary key (on MySQL/MariaDB, foreign keys works as expected with just primary key here).

Am I missing something?

Best regards :)

You're welcome to override the default foreign-key index creation, by specifying `index=False` explicitly in the `ForeignKeyField` declaration/initialization.

You might double-check your information, however, to make sure that you actually *do* have an index on `user_id` in the referring table (`UserAccess`).
Issue: shortcuts.model_to_dict convert 0 to None in ForeignKeyField unless recurse == False
Author: duvifn
State: closed
Created: 2019-07-23 11:52:29

```python
import peewee
from playhouse.shortcuts import model_to_dict
database = peewee.PostgresqlDatabase(
    'temp_db',
   ....
    )
database.connect()

class Group (peewee.Model):
    id = peewee.BigIntegerField(null= False, index = True, unique = True, primary_key = True)
    class Meta():
        database = database
class User (peewee.Model):
    id = peewee.BigIntegerField(null= False, index = True, unique = True, primary_key = True)
    group_id = peewee.ForeignKeyField(Group, null = False)

    class Meta():
        database = database

database.create_tables([Group, User])
Group.create(id = 0)
User.create(id = 0, group_id = 0)
print (model_to_dict(User.get_by_id(0))['group_id'] == 0) # False
print (model_to_dict(User.get_by_id(0), False)['group_id'] == 0) #True
```


Thanks!
Issue: Postgres transaction isolation level 
Author: duvifn
State: closed
Created: 2019-07-22 07:28:43

Hi,
Probably I'm missing something but I couldn't find a way to specify transaction isolation level for Postgres (default to [`Read committed`](https://www.postgresql.org/docs/9.5/transaction-iso.html) which is not sufficient for my use case - in which `Repeatable read` is needed).

Any way to workaround it (without changing postgres config or manually open transaction)?

Thanks a lot!

The underlying psycopg2 connection class is exposed as `db.connection()`, e.g.

```python
db = PostgresqlDatabase('my_app')
conn = db.connection()  # returns current conn, opening one if necessary
```

Then use the psycopg2 apis to set the isolation level:

```python
from psycopg2 import extensions
conn.set_isolation_level(extensions.ISOLATION_LEVEL_REPEATABLE_READ)
```

You can put all this in a hook provided by peewee and executed automatically when a new connection is created.

```python

class RCPostgresqlDatabase(PostgresqlDatabase):
    def _initialize_connection(self, conn):
        conn.set_isolation_level(...)
```

This hook is called whenever a new connection is opened, so its probably what you're looking for.

Added support for passing this into the PostgresqlDatabase `__init__()` method in master, will be available in the next release.

@coleifer Thanks for the quick response and the code addition!

Do you have any plan to add isolation level as a parameter to `db.atomic()` or `db.transaction()` in the future? Sometimes per-transaction config is needed rather than per-session....


Not really, it seems from the psycopg2 docs that when you set the isolation level it is for the duration of the session, so there doesn't seem to be an API that is just for the lifetime of a transaction. Because of this, I'm inclined to leave it as an exercise to the reader to implement a context-manager or whatever you need.

```python
@contextmanager
def isolation(level):
    conn = db.connection()
    orig = conn.isolation_level
    conn.set_isolation_level(level)
    try:
        with db.atomic() as txn:
            yield txn
    finally:
        conn.set_isolation_level(orig)
```

Something like this?

OK. Thanks again.

In case someone else will encounter this thread - another possible option (which i didn't try) is to use raw [`SET TRANSACTION`](https://www.postgresql.org/docs/9.5/sql-set-transaction.html) statement which should be ran before any other query in the transaction.
Issue:  what's the alternative usage of ReadSlaveModel
Author: kaka19ace
State: closed
Created: 2019-07-15 09:50:10

hi, peewee playhouse 3.x version has no longer support read_slave, 
and what's the alternative usage?

Instead of this, Peewee 3.x instead provides some really nice APIs for switching the database around at run-time.

* http://docs.peewee-orm.com/en/latest/peewee/api.html#Database.bind_ctx
* http://docs.peewee-orm.com/en/latest/peewee/api.html#Model.bind_ctx
Issue: Init db instance later
Author: danieldaeschle
State: closed
Created: 2019-07-12 22:05:07

Is there any chance to initalize the database instance later?

When i want to create a MysqlDatabase i must give the "database" parameter.
But at the point of creating the instance i don't know it.
But i have to create the instance to create the models.

A better way for me would be to define the models without depend on the db instance.
Is there a way to add the models later to the db instance?

Read the docs.

http://docs.peewee-orm.com/en/latest/peewee/database.html#run-time-database-configuration
Issue: Get the original SQL statement
Author: ztj1993
State: closed
Created: 2019-07-05 04:29:37

I execute the code:

```
sql = AdModel.select(AdModel.id).where(AdModel.status == 9,).sql()
print(sql)
```

The output is:
```
('SELECT `t1`.`id` FROM `ht_silo`.`ht_ad` AS `t1` WHERE (`t1`.`status` = %s)', [9])
```

How to output:
```
SELECT `t1`.`id` FROM `ht_silo`.`ht_ad` AS `t1` WHERE (`t1`.`status` = '9')
```

Thank you!

Peewee issues parameterized queries. All the db drivers peewee works with out-of-the-box support this, so the `.sql()` method returns the query and the params, which is exactly what is sent to the db.

If you want to try, you can use `peewee.query_to_string(q)` which will interpolate the parameters, but this is obviously vulnerable to sql injection and is only provided for debugging.
Issue: Matching peewee with vuex-orm
Author: dsmurrell
State: closed
Created: 2019-07-04 10:16:32

I realise that it's not your problem to match peewee with random frontend tools... however, I'm wondering whether there is a simple solution that already exists but which I don't know about that can adequately match what peewee returns with what this specific frontend tool wants.

For models Studies and Surveys, where a survey belongs to a study, if I used the code:

`jsonify({'surveys': list(query.dicts())}`

to return a list of surveys, or 

`survey = query[0]`
`jsonify({'survey': model_to_dict(survey, recurse=False)})`

to return a single survey...

The key which represents the study in the dictionary is 'study', but in both instances (in the second because `recurse=False` is used) the value is the study's id. For example:

`{
      "description": "The is the Skin Pre-survey",
      "id": "r7mzY2A8uMZv4R67Ca9tFF",
      "study": "UQMKsya34CyKWMA7Y4ZWpX",
      "title": "Skin Pre Survey"
    }`

Is there any way to configure peewee to return a key of 'study_id' instead of 'study' here? As this would mean that the naming on the client side orm can be 'study_id' for the id, and 'study' for the related study. i.e. I can say survey.study_id and survey.study in the client side code which is far more readable.

Currently, to get around this we've mapped the id onto study and the object onto study_obj and this causes confusion.

I realise that we could get around this with bit of transforming at any point in the chain, but this would require extra lines of code for each foreign key in our schema and be an additional maintenance issue.

If this functionality doesn't exist already, we're happy to add it if you think it useful. Where about would be the best place to add this?

`model_to_dict` is a convenience helper, but people definitely tend to treat it as the be-all/end-all of putting their model data in a dict.

You can try excluding the `study` field and specifying `extra_attrs=['study_id']` which will use the built-in foreign-key object ID accessor. That may work.

Thanks, I'll give that a go!
Issue: Can peewee create relations but without foreign key constraint?
Author: ethanlinys
State: closed
Created: 2019-07-04 07:33:37

In sqlalchemy I can do like below. [https://stackoverflow.com/questions/37806625/sqlalchemy-create-relations-but-without-foreign-key-constraint-in-db](url)

```
In [2]: class A(Base):
   ...:     a_id = Column(Integer, primary_key=True)
   ...:     __tablename__ = 'a'
   ...:     

In [3]: class C(Base):
   ...:     c_id = Column(Integer, primary_key=True)
   ...:     a_id = Column(Integer)
   ...:     __tablename__ = 'c'
   ...:     a = relationship('A', foreign_keys=[a_id],
   ...:                      primaryjoin='A.a_id == C.a_id')
```
How can I do this by peewee?

I don't really understand, nor am I familiar w/SQLAlchemy, so the code example makes no sense to me.

In short, if you want a foreign-key, use the foreign key field.

If not, use an `IntegerField` and add a property to your model class.

```python
class Tweet(Model):
    user_id = IntegerField()

    @property
    def user(self):
        return User.get(User.id == self.user_id)
```

But I have no idea why you would want to do this.

Yes, you can.

`myfield = peewee.ForeignKeyField(OtherModel, lazy_load=True, _deferred=True)`

I can confirm `_deferred` does not generate shit like

```
 FOREIGN KEY (`myfield_id`) REFERENCES `othermodel` (`id`)
```
Issue: Postgres field extension - inet data type
Author: danielguardicore
State: closed
Created: 2019-07-03 16:23:50

First off, Peewee is great :)

Postgres supports network data types.
https://www.postgresql.org/docs/9.1/datatype-net-types.html
What would be the best way to implement support for these fields instead of peewees IPAddr field?

A simple Field extension with the right type would suffice?

If so, I will prototype it and open a PR

```python
class SomeField(Field):
    field_type = 'inet'
```

If you require conversion from python data-types to db-friendly data-types, you would implement `python_value()` and `db_value()` as needed.

http://docs.peewee-orm.com/en/latest/peewee/models.html#custom-fields
Issue: Missing on= inference for joins (ValueError: More than one foreign key between.... Please specify which you are joining on.)
Author: nad2000
State: closed
Created: 2019-07-03 02:54:27

The inference of **on=...** has changed in the case with multiple foreign keys to the same table, for example, 

```python
class User(Model):
    ...

class Record(Model):
   user = ForeignKeyField(User)
   created_by = ForeignKeyField(User)
   updated_by = ForeignKeyField(User)
```

an attempt to join `Record` to `User` without given **on=...** (`python Record.select().join(User)`)  fails with:
```
ValueError: More than one foreign key between <Model: Record> and <Model: User>. Please specify which you are joining on.
```

I reckon it used to pick up, by default, the foreign key with the name that matches the calls name, namely, **user** in this example.



My sense is to keep this explicit when multiple FKs are present, regardless of what the previous (2.x?) behavior may have been.

that's a bummer. I have heaps of queries that used to work with 2.x but now are broken (all tables have these created_by and modified_by)

Well let me reconsider.

Merged equivalent patch thanks.
Issue: Can't access to joined model data
Author: kyotops
State: closed
Created: 2019-07-02 21:31:58

I'm trying to implement Many to Many select (http://docs.peewee-orm.com/en/latest/peewee/relationships.html#implementing-many-to-many), but I can't access to Book model data:
```
query = User.select(
    User,
    Book,
    BookRelationship,
).join(
    BookRelationship,
    on=BookRelationship.from_user,
).join(
    Book,
).where(
    BookRelationship.to_user == self,
).order_by(
    BookRelationship.created_at.desc()
)

for item in query:
    # Can access only user data (item.username, item.fullname, item.age)
```
In this example, I'm trying to retrieve user data along with books he liked.

I didn't want to open this issue, because of this kind of trifle, but I couldn't find another way out.

In a many-to-many things are complicated. Depending on how you join, you're going to get duplicated data on one side or the other. Remember that the output of any query is always going to be a flat table-like data-structure. There is no nesting. So no rolling up of multiple Books for a single User. You'll get one User and one Book per User `times` Book.

You can still access this, you'd probably do something like

```python
for user in query:
   related_book = user.book_relationship.book  # Something like this, depends on your field names.
```

Your best bet is to do something like this:

```python
user = User.get(User.username == 'charlie')
liked_books = Book.select().join(BookRelationship, on=BookRelationship.from_user).where(BookRelationship.to_user=user)
```
Issue: .exists() fails for a query with SQL literals/expresions 
Author: nad2000
State: closed
Created: 2019-07-02 03:38:52

With **2.x** something like this was working.
```python 
# This is a simplified query (check in one round-trip):
(User.select(SQL('1')) | Organisation.select(SQL('1'))).exists()
```

With **3.x (3.9.6)** the query gets build but **.exists()** fails with:
```
*** AttributeError: 'ModelCompoundSelectQuery' object has no attribute 'columns'
```

```sql
-- (User.select(SQL('1')) | Organisation.select(SQL('1'))).sql():
SELECT 1 FROM "user" AS "t1" UNION SELECT 1 FROM "organisation" AS "t2"
```

It might be that it has nothing to do specifically with SQL expressions
```python
(User.select(User.id) | Organisation.select(Organisation.id)).exists()
```
also fails with
```
*** AttributeError: 'ModelCompoundSelectQuery' object has no attribute 'columns'
```
Issue: Add support for MySQL STRAIGHT_JOIN
Author: d0d0
State: closed
Created: 2019-06-19 11:57:08

Add support for mysql STRAIGHT_JOIN. Resolves https://github.com/coleifer/peewee/issues/842

Going to pass.

Is the feature available in the peewee now ?

I see latest peewee version was added in Apr 23, 2020, so this one not yet released ???

I did not merge this change.

Any alternative way to include straight_join in peewee ?

or is there a way to replace **select** with **select straight_join** once the query is built ? any solution related to this one is much appreciated thanks

It seems like a very obscure mysql thing to try and hint the query-planner to use a certain ordering. I don't really plan on supporting or maintaining that functionality. I'd suggest tweaking your query or re-generating statistics.

okay
thanks
Issue: How to add fields to existing tables
Author: qa1057
State: closed
Created: 2019-06-19 08:05:52

For example, now I have created a model and executed to create a table. Then I want to add fields to this table. What should I do?

* https://www.postgresql.org/docs/current/sql-altertable.html
* https://www.sqlite.org/lang_altertable.html
* https://dev.mysql.com/doc/refman/8.0/en/alter-table.html

Peewee can help you. If you had looked at the docs, you'd probably have found it:

http://docs.peewee-orm.com/en/latest/peewee/playhouse.html#migrate

This kind of question belongs on stack overflow. NOT on the issue tracker. Please use your brain.
Issue: Implement full-text search in mysql?
Author: dzpt
State: closed
Created: 2019-06-18 17:49:52

It wasn't mentioned on the doc. Only issue #532 
You link to user-defined function but i have no idea how to use it to implement full-text search.
There's no doc on `Expression`
I've tried with this, but how am i supposed to set MATCH around `lhs`?
```
def fulltext(lhs, rhs):
    return Expression(lhs, 'AGAINST', rhs, True)
```

I have to use raw query like this instead
`q = q.select().where(SQL("MATCH (title) AGAINST(%s)", ("test",)))`

have no idea why i have to use `("test",)`, if i input `"test"`:
`q = q.select().where(SQL("MATCH (title) AGAINST(%s)", "test"))`

i'll get `TypeError: not all arguments converted during string formatting`

Here is the doc: http://docs.peewee-orm.com/en/latest/peewee/query_operators.html#adding-user-defined-operators

Just write a wrapper.

```python
def Match(columns, expr, modifier=None):
    match = fn.MATCH(*columns)  # MATCH(col0, col1, ...)
    args = expr if modifier is None else NodeList((expr, SQL(modifier)))  
    return NodeList((match, fn.AGAINST(args)))

.where(Match((Post.title, Post.text), 'search query'))

.where(Match((Post.title, Post.text), 'foo AND bar', 'IN BOOLEAN MODE'))
```

To make it even easier, I added an implementation in f76ce0b.
Issue: After importing  database, IntegrityError raised for every new record
Author: joshuaia
State: closed
Created: 2019-06-16 14:27:30

I'm importing database like this:
`psql -h localhost -d db_name -U user -f db.sql`

Example of sql query:
```
INSERT INTO public.users(id, email, password) VALUES (1, 'user1@mail.com', 'PASSWORD_HASH');
INSERT INTO public.users(id, email, password) VALUES (2, 'user2@mail.com', 'PASSWORD_HASH');
INSERT INTO public.users(id, email, password) VALUES (3, 'user3@mail.com', 'PASSWORD_HASH');
```

User model:
```
class User(db.Model):
    email = CharField(unique=True)
    password = CharField()
```

After that, I'm trying to create new record, eg. register new user:
```
new_user = User.create(
            email=email,
            password=generate_password_hash(password),
        )
# After creating, redirect user to home page
```

But at this moment `IntegrityError` is raised:
`duplicate key value violates unique constraint "users_pkey" DETAIL: Key (id)=(1) already exists.`

But, without importing the DB, the error does not appear. And new data is successfully saving.

You need to update the sequence that controls the next value of the primary key.

```sql
SELECT setval('users_id_seq', (SELECT MAX(id) FROM users)+1);
```

For whatever reason, the way you're doing your inserts isn't setting this properly.