MovingBlocks/Terasology — pull request #391 by mbrotz (2012-12-18 13:48:29+00:00)
Title: Sparse chunks

Hi all

In the past two weeks or so i was working on sparse chunks. The current implementation is working with all terrain generators and gives a very signifficant reduction in memory consumption.

Most chunks can be compressed by about 80 to 90 percent. And on my machine i have no signifficant impact on performance!

The implementation is based on four important classes:
* TeraDenseArray8Bit --> TeraSparseArray8Bit
* TeraDenseArray4Bit --> TeraSparseArray4Bit

When a chunk is initially generated it uses the dense arrays for performance reasons. So chunk generators run without performance loss.

As soon as the state of the chunk is set to COMPLETE, the chunk switches to the sparse arrays and throws away as much redundant data as possible.

This works for block, sunlight, light and liquid data. Especially light and liquid data can be reduced by up to 99% in most cases.

At runtime the sparse arrays are extended as necessary when the world is changed.

There is still some tuning possible. Currently, chunks are compressed only once immediately after their state was set to COMPLETE. After that, they never get recompressed.

Also, if a sparse array is completely filled with non-redundant data it uses more memory than its dense version. Switching back to its dense version in such a case would be desirable.

Further, all the classes are designed to make it easy to add alternative implementations in the future. It's also easy to switch to a bigger datatype for block id's in the future.

And of course every time a chunk is compressed it generates a nice log message to indicate by how much its data has been reduced. :)

Please take a look at it and give me some feedback.

Panserbjoern