bitcoinjs/node-bitcoin-p2p — Issue #55: "Too many open files" when using LevelDB
Author: justmoon
Created: 2011-12-28 04:54:04+00:00
State: open

When downloading the full block chain, you may run into a "Too many open files" error.

This is related to the number of LevelDB databases we run concurrently and not a problem with LevelDB itself.

Please post a comment here if you run into this issue. Post your OS and version and, if you know it, your open files limit. (`ulimit -Sn` to find out the limit on Linux)

This is fixable, I just need to be careful not to reduce performance by setting file descriptor limits that are too low.

As a workaround you can [raise your open files limit](http://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/) or simply restart the node (it should only run into this error during the initial download and only after downloading tens of thousands of blocks at once.)

The current theoretical maximum number of file descriptors that the LevelDB back end will consume is 7000 (seven databases, 1000 open files per database.)

Two main approaches to fixing this:

1. Reduce the maximum open files limit for each database to bring the total under 1000. This would be just a few lines of code and may or may not impact performance noticeably.

2. Store everything in a single database. This is possible to do by simply using prefixes for each table. We can do our own accounting of database meta-information in a special record (e.g. "meta"). This is probably the best solution although we have to check how well LevelDB handles uneven load.