Unique Sorting in Solidity using Value Arrays
By: Jules Goddard
Tags: Blockchain, Ethereum, Smart Contracts, Programming, Solidity

Unique Sorting in Solidity using Value Arrays

This article considers using value arrays to improve sorting in Solidity, the defacto smart contract language for the Ethereum blockchain.

Photo by the author

Background

In my article “Sorting in Solidity without Comparison”, I compared various sorting techniques on Solidity memory arrays. In summary, my conclusion, according to the provided code, data and tests, was that the Unique Sort on dynamic uint memory arrays was most suitable for my particular application. If you need to sort Solidity storage arrays, consider the article “Shadowing Solidity Storage Variables in Memory” which illustrates the best technique.

Following my further articles “Value Arrays in Solidity” and “Dynamic Value Arrays in Solidity”, I revisit the Unique Sort algorithm here, to determine whether using value arrays can improve performance further.

Discussion

A Value Array is simply an array held in a value type, as opposed to a reference array. Solidity runs on the Ethereum Virtual Machine (EVM) which has a very large machine word of 256bits (32bytes). It is this latter feature that enables us to consider using Value Arrays.

Here is the Unique Sort algorithm from the sorting article above:

function unique(uint[] memory data, uint setSize) internal pure {

uint length = data.length;

bool[] memory set = new bool[](setSize);

for (uint i = 0; i < length; i++) {

set[data[i]] = true;

}

uint n = 0;

for (uint i = 0; i < setSize; i++) {

if (set[i]) {

data[n] = i;

if (++n >= length) break;

}

}

}

This generally sorts unique data using the least gas, according to my measurements of Quick Sort, Insertion Sort, Counting Sort and Unique Sort algorithms on dynamic uint arrays of lengths 10, 13 and 17 elements, whether the code is non-optimised or optimised:

Measuring gas cost of different Sorting Algorithms and array lengths

The Unique Sort data is represented by the red columns.

There are 2 aspects worth considering for improvements in the Unique Sort algorithm.

If the data set is small enough (setSize not more than 256 elements), we could possibly use a value array to represent the booleans indicating the presence or absence (the inclusion) of a number in the set. The data is an array of uints, and so we could possibly use a value array to represent the data. Value arrays are great for reducing gas consumption in storage and memory, but will they be helpful when there is a lot of accessing and swapping individual elements?

Value Array Representing the Inclusion Set

The code is trivially refactored as:

function unique(uint[] memory data, uint setSize) internal pure {

require(setSize <= 256);

uint length = data.length;

uint set;

for (uint i = 0; i < length; i++) {

set |= 1 << data[i];

}

uint n = 0;

uint mask = 1;

for (uint i = 0; i < setSize; i++) {

if ((set & mask) != 0) {

data[n] = i;

if (++n >= length) break;

}

mask <<= 1;

}

}

I compiled the test harness, described in “Gas Cost of Solidity Functions” and the test code using Solidity Istanbul v0.6.12.

And here is the effect on the performance compared with my previous measurements:

Measuring gas cost of different Sorting Algorithms and array lengths

Quite a helpful change, the Unique Sort function now consumes approximately 20% less gas.

Value Array Representing the Data

This is a much more complex change, also requiring considerable changes to the test harness. We use a bytes32 type to contain the array values, since that has a compiler facility to read individual bytes using array-style indexing, such as data[i], although we still need to provide a function to set individual bytes:

function sortUnique(bytes32 data, uint length, uint setSize) internal pure returns (bytes32) {

uint set;

for (uint i = 0; i < length; i++) {

set |= 1 << uint(uint8(data[i]));

}

uint n = 0;

uint mask = 1;

for (uint i = 0; i < setSize; i++) {

if ((set & mask) != 0) {

data = bytes32lib.set(data, n, byte(uint8(i)));

if (++n >= length) break;

}

mask <<= 1;

}

return data;

}

The bytes32lib.set() library function sets the nth byte in the value array. It is provided in the article “Value Arrays in Solidity”. And here is the effect on the performance:

Measuring gas cost of different Sorting Algorithms and array lengths

Wow! This shows just how much gas those bit-fiddling library functions consume. The modified Unique Sort function is 8 times slower than simple assignment.

Conclusions

I have provided and measured code for sorting small unique sets of numbers in Solidity bytes32 variables.

Yes, we can reduce our gas consumption using a Value Array for the inclusion set.

Where our Solidity smart contracts carry out a large number of array accesses, for example during sorting, then the use of the library functions required for Value Arrays is likely to consume more gas than reference arrays.

Bio

Jules Goddard is a founder of Datona Labs, who provide smart contracts to protect your digital information from abuse.

website — twitter — discord — GitHub

Also, Read