andymccurdy/redis-py — Issue #205: MemoryError when getting large values.
Author: KeirRice
Created: 2011-10-21 01:15:42+00:00
State: open

Hi All,

I have been getting a MemoryError (partial stack below) when the server returns a large value. In my case the value was on the order of 20MB. I suspect that the issue maybe the windows socket implementation.

Some searching turned up a similar issue in 'imaplib': http://bugs.python.org/issue1441530
Their solution was to work around it by only requesting 10MB chunks from the socket at a time and buffer the results.

```python
File "c:\python25\lib\site-packages\redis-2.4.9-py2.5.egg\redis\client.py", line 384, in get
return self.execute_command('GET', name)
File "c:\python25\lib\site-packages\redis-2.4.9-py2.5.egg\redis\client.py", line 253, in execute_command
return self.parse_response(connection, command_name, **options)
File "c:\python25\lib\site-packages\redis-2.4.9-py2.5.egg\redis\client.py", line 263, in parse_response
response = connection.read_response()
File "C:\Python25\lib\site-packages\redis-2.4.9-py2.5.egg\redis\connection.py", line 222, in read_response
response = self._parser.read_response()
File "C:\Python25\lib\site-packages\redis-2.4.9-py2.5.egg\redis\connection.py", line 80, in read_response
response = self.read(length)
File "C:\Python25\lib\site-packages\redis-2.4.9-py2.5.egg\redis\connection.py", line 45, in read
return self._fp.read(length+2)[:-2]
File "C:\Python25\lib\socket.py", line 309, in read
data = self._sock.recv(recv_size)
MemoryError
```