Issue: How to use the read and write with multithreading?
Author: ismkv
State: closed
Created: 2015-01-07 19:29:06

Good day. Please help me with a problem.

I'm having trouble updating / reading data in a thread when using ORM.

I've been using peewee for Python. After reading the documentation starting to initialize the object to work with the database.

``` python
db = SqliteDatabase('db.sqlite', threadlocals=True)

# Next comes the creation of tables and filling.
class Products(Model):
    id = IntegerField(index=True, primary_key=True)
    count_sales = IntegerField()
    updated = IntegerField() # default to zero

    class Meta:
        database = db

db.connect()
db.create_tables([Products, ...]) # I have several tables

# I continue to fill the table with data from a file
```

I then create a predetermined amount of threads as follows:

``` python
class MyThread(Thread):
    tables = None
    db = None

    def run(self):
        products_update(self.tables, self.db)

for _ in range(1, 20):
    t = MyThread()
    t.tables = (Products, ...)
    t.db = db
    t.start()
```

The function counts the number of records in the field `count_sales`, and there they are, it changes randomly one of them.

``` python
def products_update(tables, db):
    Products, ... = tables

    count = Products.select().where(Products.count_sales < 10, Products.updated == 0).limit(1).count()
    if count:
        row = Products.select().where(Products.count_sales < 10, Products.updated == 0).order_by(random()).limit(1).get()

        Products.update(updated=1).where(Products.id == row.id).execute()
```

I need to make the process of finding and updating was linear.
**Thank you for your attention. I apologize for the language I used a translator.**


I'm not exactly sure I understand what problem, specifically, you are encountering.


@coleifer, I made an example for you to understand:

``` python
from peewee import *
from threading import Thread
from random import random, randint

import os

DB_FILE = 'db.sqlite'

if os.path.isfile(DB_FILE):
    os.remove(DB_FILE)

db = SqliteDatabase(DB_FILE, threadlocals=True)


# Next comes the creation of tables and filling.
class Products(Model):
    id = IntegerField(index=True, primary_key=True)
    count_sales = IntegerField()
    updated = IntegerField() # default to zero

    class Meta:
        database = db

db.connect()
db.create_tables([Products]) # I have several tables

data = []
for _ in range(0, 40):
    data.append({'count_sales': randint(2, 9), 'updated': 0})

with db.transaction():
    for data_dict in data:
        Products.create(**data_dict)

print('Count products: {}\n'.format(Products.select().count()))


def products_update(n, tables, db):
    Products, = tables

    count = Products.select().where(Products.count_sales < 10, Products.updated == 0).count()
    print('Thread #{}, {} records found'.format(n, count))
    if count:
        row = Products.select().where(Products.count_sales < 10, Products.updated == 0).order_by(random()).limit(1).get()

        Products.update(updated=1).where(Products.id == row.id).execute()


class MyThread(Thread):
    n = None
    tables = None
    db = None

    def run(self):
        products_update(self.n, self.tables, self.db)

for n in range(1, 5):
    t = MyThread()
    t.tables = (Products,)
    t.db = db
    t.n = n
    t.start()

# Result:
#   --------------------
#   Count products: 40
#
#   Thread #1, 40 records found
#   Thread #2, 40 records found
#   Thread #3, 40 records found
#   Thread #4, 39 records found
```


And I need the result to be consistent:

```
#   Thread #1, 40 records found
#   Thread #2, 39 records found
#   Thread #3, 38 records found
#   Thread #4, 37 records found
```


I appreciate you're spending the time creating this issue, but this isn't the appropriate place for this type of question. Try StackOverflow?


@coleifer Tried, they could not help. Can you tell me where to dig?


I'm failing to see how this is a peewee issue and not an issue of incorrect use of locks or incorrect assumptions about concurrent behavior.