The payout of a mining pool is influenced by lots of variables like network difficulty, total hashrate of the pool, pool server latency, payout method, fees, pool luck and many more. Comparing systems with so many changing variables is not trivial and needs a special approach. The naive method to compare mining pools is to select a few candidates for a coin and mine x days on the first, then x days on the second and so on. After having mined on all candidate pools you compare how much coins you have received from each pool. This method has a key problem as it does not take into account changes in the network difficulty of the coin. While mining on the first pool the difficulty could have been higher or lower compared to when mining on the other pools. Especially with young coins one can expect large difficulty variations. Also if x is short pool luck will influence the results significantly. Another method is to mine on all tested pools at the same time using multiple GPUs/ASICS. While this method will yield much better results than the first approach our experience shows that it still has its problems. First if the tested period is to short, pool luck will still influence the results. Furthermore, even if two rigs are built from the same components, their hashrate might not be to 100% similar and will still be subject to large variance. If we want to detect pool performance differences in the range of 1–2% such a hashrate difference will also have an significant impact on the results.