aarddict/tools — Issue #17: mmap.error: [Errno 12] Cannot allocate memory when starting to compile wiki dictionary
Author: oha59
Created: 2012-03-01 21:40:06+00:00
State: open

After processing of the CDB-Files from dewiki is finished I got an error when aardc is about to start compiling the aar files.

100.00% t: 2 days, 16:11:42 avg: 10.0/s a: 1373941 r: 946049 s: 0 e: 0 to: 0 f: 0
Compiling .aar files
Traceback (most recent call last):
File "/home/oliver/env-aard/bin/aardc", line 9, in
load_entry_point('aardtools==0.8.3', 'console_scripts', 'aardc')()
File "/home/oliver/env-aard/lib/python2.6/site-packages/aardtools/compiler.py", line 1094, in main
compiler.compile()
File "/home/oliver/env-aard/lib/python2.6/site-packages/aardtools/compiler.py", line 507, in compile
for volume in self.make_volumes(create_volume_func, articles):
File "/home/oliver/env-aard/lib/python2.6/site-packages/aardtools/compiler.py", line 526, in make_volumes
for title, serialized_article in articles:
File "/home/oliver/env-aard/lib/python2.6/site-packages/aardtools/compiler.py", line 388, in sorted
article_store = mmap.mmap(article_store_f.fileno(), 0)
mmap.error: [Errno 12] Cannot allocate memory

Compilation of an older dewiki dump finished successfully a couple of days ago.

Now I have the following files in the aardc working directory:
-rw------- 1 oliver oliver 38905125 2012-03-01 14:38 aa-8x_Ux4.titles
-rw------- 1 oliver oliver 3374091660 2012-03-01 14:38 aa-Pg6S6z.articles
-rw------- 1 oliver oliver 41759820 2012-03-01 14:38 aa-R5GzZO.index

Processing took two days an 16 hours, so I don't feel like doing it again.
Free disk space is 7,7 GB, but I'm not sure, if that is the problem.
Is there a way to start the compilation of the .aar files using the titles, articles and index files without the need to create them again from the CDB files?