Cairnarvon/progscrape — Issue #7: MemoryError
Author: johnsmith123
Created: 2011-11-15 07:14:06+00:00
State: open

```
←[1AScraping... [#### ] 21.54% (3858/17912)
←[1AScraping... [#### ] 21.54% (3859/17912)
←[1AScraping... [#### ] 21.55% (3860/17912)
←[1AScraping... [#### ] 21.56% (3861/17912)
←[1AScraping... [#### ] 21.56% (3862/17912)
←[1AScraping... [#### ] 21.57% (3863/17912)
Exception in thread Thread-19:
Traceback (most recent call last):
File "C:\Python27\lib\threading.py", line 552, in __bootstrap_inner
self.run()
File "C:\Python27\lib\threading.py", line 505, in run
self.__target(*self.__args, **self.__kwargs)
File "C:\Users\johnsmith123\Desktop\progscrape.py", line 394, in scrape_json
page = json.loads(page)
File "C:\Python27\lib\json\__init__.py", line 326, in loads
return _default_decoder.decode(s)
File "C:\Python27\lib\json\decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python27\lib\json\decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
MemoryError

Exception in thread Thread-7:
Traceback (most recent call last):
File "C:\Python27\lib\threading.py", line 552, in __bootstrap_inner
self.run()
File "C:\Python27\lib\threading.py", line 505, in run
self.__target(*self.__args, **self.__kwargs)
File "C:\Users\johnsmith123\Desktop\progscrape.py", line 394, in scrape_json
page = json.loads(page)
File "C:\Python27\lib\json\__init__.py", line 326, in loads
return _default_decoder.decode(s)
File "C:\Python27\lib\json\decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Python27\lib\json\decoder.py", line 382, in raw_decode
obj, end = self.scan_once(s, idx)
MemoryError
```
(...etc)

and then python.exe crashes inside python27.dll.
This was doing an update from the 2010-08-01 database here on github, just running ./progscrape.py.
Is there some sort of memory leak?

_______________________________
Running 32-bit python 2.7.2 on Windows 7 x64 with 4GB ram.