Simulation of Incentive Design: What is the Most Appropriate Reward System? Part 1
Tags: Agent Based Modeling, Simulation, Reinforcement Learning, Blockchain, Machine Learning

How to design a reward system?

The next topic of Decon Simulation series is ‘reward system design’. We will look into what problems may spawn from reward system design before diving into the simulation.

Where Will the Rugby Ball Bounce to?

Behavioral Diversity

On a traditional blockchain, users have limited scope of behavior and rewards are automatically given without human intervention. For instance, only through mining can you receive a reward on Bitcoin. The reward is automatically given in the form of coinbase transaction to the person who solved a problem.

Such behavior simplicity and fully automatic rewarding makes it relatively easy to predict and drive user action. In other words, it is easy to design a reward system because the design can be built purely with math.

However, on blockchains where rewards are given to various activities other than mining, the reward system design becomes complex. For example, if users are motivated to upload posts (Steemit), propose new algorithms for certain problems (Numerai), or post good data set (Ocean), how should the designer distribute rewards?

Reward System Problem

Because it is impossible to complete automate the process of assessing whether a user made a meaningful contribution, the blockchain needs the help of other users for reward distribution. Many token economies solve this problem through voting. In a voting reward system, the post that get more likes by users receive more rewards.

For example, a system seeks to give appropriate rewards to users for their postings. An appropriate reward is assessed by the post’s quality and reader response. Because such assessment requires quality check, automation is very difficult. Therefore, the number of likes a post receives becomes the reward criteria. In this case, pressing the like button is equivalent to voting, and the aggregate number of likes is the number of votes.

It would seem reasonable to receive an award equal to the percentage of likes a person’s post got out of all the likes from the ‘reward pool’, which is the aggregate total of rewards. However, could such a system continuously drive high quality activities?

Methods of Giving Rewards

At a glimpse, it would be reasonable to give rewards proportionally based on the number of likes a person receives out of all likes.

However, some users could refuse to post anything for reasons unrelated to effort (popularity, exposure of skin, etc.). In an environment with such users, a uniform reward structure is necessary.

Also, if there are too many users (where the total reward is capped), a reward could shrink even though a user managed to attract relatively many likes. To resolve this problem, a reward system that increases exponentially based on the number of likes should be established to incentivise high quality posts.

There is no best reward system, only the most appropriate.

Then, which out of proportional, uniform, and exponential reward systems is the best? Unfortunately, the answer to that question is “none”. There is no cure-all system that can be applied in all cases.

here can be the most suitable system for a certain environment. An exponential reward system would be a good fit for an environment with many users while a uniform system would be better suited for an environment with few users.

In truth, this problem would be difficult to tackle before an environment is created and functional. Yet, it is an arduous task to overhaul a design once a service is up and going. Decon experimented on a simulation to see what the optimum reward distribution system is.

Nobody can anticipate where a rugby ball will bounce off to, but if reality is effectively incorporated into simulation, we can get a rough estimate of the trajectory. Similarly, even though individual behavior is diverse, we can roughly model patterns with simulation. We aimed to come up with a better reward system.