Issue: peewee mysql gone away error
Author: FashtimeDotCom
State: closed
Created: 2014-05-09 01:09:13

when i use peewee，and try to fix this error, but try more ways failed! the error always occured!

exception::

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/tornado/web.py", line 1141, in _when_complete
    callback()
  File "/usr/local/lib/python2.7/dist-packages/tornado/web.py", line 1162, in _execute_method
    self._when_complete(method(_self.path_args, *_self.path_kwargs),
  File "/home/sch/cakeshop/src/handler/site.py", line 17, in get
    for shop in Shop.select(Shop.name, Shop.ename, Shop.cover, Shop.price).where((Shop.cid != 2) & (Shop.status != 9)).order_by(Shop.views.desc()).limit(6):
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 2086, in __iter__
    return iter(self.execute())
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 2079, in execute
    self._qr = ResultWrapper(model_class, self._execute(), query_meta)
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 1796, in _execute
    return self.database.execute_sql(sql, params, self.require_commit)
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 2359, in execute_sql
    self.commit()
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 2230, in __exit__
    reraise(new_type, new_type(*exc_value.args), traceback)
  File "/usr/local/lib/python2.7/dist-packages/peewee.py", line 2351, in execute_sql
    cursor.execute(sql, params or ())
  File "/usr/local/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "/usr/local/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
OperationalError: (2006, 'MySQL server has gone away')


This is not a bug with peewee per-se. This has been brought up a few times.
- #225
- #239 
- #67 

Please search before creating new issues in the future!


## http://docs.peewee-orm.com/en/latest/peewee/database.html#error-2006-mysql-server-has-gone-away


An alternate fix is implemented in 017e4e4952f0b429dd21f0e90465a0f644cf6015


Hi @coleifer: Apologies for re-addressing this closed issue, which I see you have addressed in numerous places. I have reviewed the issues and StackOverflow, and the error persists for me.

I am using a single connection (not a pool), and using the connection context manager, as described in the PeeWee documentation. The data volume is small (~100KB per row), and the problem only arises with using `self.model.insert_many(rows).execute()`. Writing a small number of rows at one time works (~1-30 rows), and debugging shows that the connection object displaying the expected behavior: the connection is closed prior to entering the context manager, open during the context manager, and closes thereafter. Writing 30+ rows fails with the 2006 error. 

```
        batch_size = 100
        logger.info(db_to_use.is_closed())
        with db_to_use.connection_context():
            logger.info(db_to_use.is_closed())
            with db_to_use.atomic():
                logger.info(db_to_use.is_closed())
                for idx in range(0, len(row_data), batch_size):
                    logger.info(db_to_use.is_closed())
                    rows = row_data[idx:idx + batch_size]
                    self.model.insert_many(rows).execute()
        logger.info(db_to_use.is_closed())
```

Alternatively, using `batch_commit` does not create a 2006 error, regardless of batch size.
```
batch_size = 100
logger.info(db_to_use.is_closed())
with db_to_use.connection_context():
    logger.info(db_to_use.is_closed())
    with db_to_use.atomic():
        logger.info(db_to_use.is_closed())
        for row in db_to_use.batch_commit(row_data, batch_size):
            logger.info(db_to_use.is_closed())
            self.model.create(**row)
logger.info(db_to_use.is_closed())
```

And, committing without batches also works:
```
logger.info(db_to_use.is_closed())
with db_to_use.connection_context():
    logger.info(db_to_use.is_closed())
    with db_to_use.atomic():
        logger.info(db_to_use.is_closed())
        for row in row_data:
            logger.info(db_to_use.is_closed())
            self.model.create(**row)
logger.info(db_to_use.is_closed())
```

For now, I can workaround this issue by using `batch_commit`. It would be helpful to understand why `insert_many` has this behavior. Thanks.

I can't explain it, no idea. All this stuff is unit-tested across python versions and supported databases (sqlite/mysql/python), so I'm confident there's nothing fundamentally broken in the implementations. Did you try committing after each round of `insert_many` with success? Are you running out of memory because of a large transaction and mysql is closing the connection? You might check your server logs to see why mysql is dropping you.