What is the nonce?

Let me explain what the nonce is, in mining. We have talked about cryptographic hash functions, which produce a specific fingerprint [of] data, such that if you make a tiny change to the data, it will change the fingerprint, allowing you to determine that the data has changed. The hash [could be] a completely different number even if you only change one bit in the input. Essentially, what hash functions do is take all the bits of your data and stirs them around algorithmically. If you look at how the algorithm works, it adds up binary numbers with a modulo. If you add up to more than 32 bits, it wraps around and starts again, which mixes those bits. There are also binary shift operations where the algorithm will shift a chunk of bits to the left or right, and then add them to other bits. It is [similar to] shuffling cards. The data that goes in to the cryptographic hash algorithm... is cut into chunks and then mixed together, until you eventually [produce] a 256-bit number as the output. Your cryptographic hash is the fingerprint. If one of those bits of data is different, it causes the mixing process to go down a different path, which produces a completely different fingerprint. Just changing one bit changes the cryptographic hash function's output, which is effectively a random number. It is a random number in that you can't predict what it will be. But it is also a deterministic number, because if you input the same data, the algorithm will give you the exact same fingerprint every time. So it would be very easy for anyone to verify the data. If you put the same data through the algorithm [again], you will get the same fingerprint as everybody else who hashes it. This is a very interesting trick. Changing the data even by a tiny bit will seemingly produce... a completely different random number; if you present the same data to someone else, and they put it through the hash function, it will produce the same fingerprint. What if you set a target? What if I want my fingerprint to start with a zero? What are the chances it will? If you are considering it in binary [terms], the first digit can either be zero or one. If you keep [generating] fingerprints, half of them will start with zero and half will start with one. You have a 50 / 50 chance. What if you wanted both of the first two binary digits to be zeros? In that case, you have a 25% or 1 / 4 chance that the first two binary digits will be zeros. If you keep generating fingerprints, hashing and hashing, one out of four will start with two binary zero digits. If you keep going and require [another] zero, you will have half as many hashes which fit that target. [Let's say that you require] thirty zeros. Only one in a quintillion hashes will [begin] with thirty zero digits. In order to find that [hash], you must modify your block data slightly, calculate the hash, and see if it has thirty leading zeros. It [probably] won't. Try again a quintillion times until you produce a hash that starts with thirty zeros. What is the nonce? The nonce is the part of the block that you are changing. The block header contains a timestamp, block number / height, and the Merkle root, which is like a fingerprint of all the transactions in the block. It contains the hash of the previous block. For a block to be valid, this information cannot change. You can change the timestamp by a few seconds, but the rest of this data can't be changed. The previous block and fingerprint of transactions will stay the same in the candidate block you built. There is a special 32-bit part of block that doesn't do anything; you can put any number you want in there. Why does that space exist it? It exists to allow you to change the block header and produce a different hash. We call that the "nonce." Let's say you [try to generate a block hash] with a certain number of leading zeros. Does it have the necessary number of zeros in the beginning to meet your target? No, try again. As you keep incrementing that 32-bit number, every time you [generate a block] hash, it is a different fingerprint. You need [to generate about] five quadrillion [hashes]. Eventually, you will produce a hash which starts with [enough] zeros by chance. Then you can tell everyone [in the network], "Here is a block! If you put this nonce in, the number 6,475,322, and [generate] the block header, it has thirty leading zeros in it." How do you know that it took me a lot of time to find? You know that the only way I could have [found it]... was by plugging [random] numbers into the nonce, [generating a block header], and checking the results. That is what a nonce is, the space in the block where you can just plug in a random number. [You must] keep trying in order to produce a fingerprint [that matches the target]. Then you [share] that random number with everybody [in the network], and they can verify it by producing... the fingerprint for the block [with the nonce], with a special pattern of zeros in the beginning. [They know] it must have [required] a lot of work to find that random number. It is proof that you did the work. It is proof-of-work. That is how the nonce [functions within mining]. There are some complications, because 32 bits is not enough space to do all the hashing you need. Miners also change some bits in the coinbase, and part of the timestamp. They need to move a lot of bits in the input. The nonce isn't only in one place, but that is an implementation detail.

Do the miners know the amount of zeros required for the next block? Difficulty target

There are a couple of follow-up questions on that, which I find interesting. Good questions in the chatroom today. In talking about nonces... "Do the miners know the amount of zeros required for the next block?" Yes. That is the difficulty target. The difficulty target is how many zeros the hash needs to have, in binary. Just think of it as a number. If I would say that a number starts with [some amount of] zeros, effectively the target number is less than that. If 'x' is the target, the hash must be a smaller number than the difficulty target, which is known by every node.. on the network; every node calculates the difficulty target using the same formula every 2016 blocks. They input the same numbers into that formula and produce the same answer. [They look at] what the difficulty target was for the previous two-week period, how many blocks have been mined since, and how long did those blocks take to mine? In two weeks, we want to mine 2016 blocks, and each block should be mined in ten minutes. Theoretically, that would take 20,160 minutes in total to mine these blocks. Well, how long did it take this time? Oh, it took 10% longer than usual. This means that miners are not able to find blocks every ten minutes, they are a bit slower. It is 10% too difficult. The difficulty needs to be reduced by 10%. The number of blocks is fixed, so everybody knows how long it took to mine them. They count it the same way. They can do division and find exactly how much to adjust the difficulty target. Everybody will know the difficulty target for the previous 2016 round, and the difficulty target in the new period. 'Based on my calculations, I expect the difficulty target for the next period will be 'x.' 'I will only accept blocks where the hash is less than 'x.' Everybody will calculate the same answer for 'x.' Everybody is [expecting] miners to achieve the same amount of proof-of-work for the next block. The difficulty retargeting happens every two weeks, or more specifically every 2016 blocks. Everyone on the network will know exactly what the difficulty should be, at any moment in time, and arrive at the same conclusion. Miners know what they need to do to succeed. If they have the target, [they evaluate], is the hash less than the target? If yes, it is a valid block. If no, keep hashing.

Is it possible to guess the nonce on the first try?

Try more nonces. Another question is, "I wonder, is it possible to guess the nonce on the first try?" Yeah, it is possible. You might find it on the first try, you might find it on the quintillionth try. It doesn't matter. The probabilities of finding it on your first, second, or third try are the same. These are independent events; the probability of each event does not affect the previous or the next event. There is [no dependency] link between them. Just as if I were to flip a coin. Whether it comes up heads or tails, the chances the next time I flip it... are still 50%, even if I flip a hundred times and it comes up tails one hundred times. On the hundredth and first flip, the probability is 50% that it will come up heads, 50% that it will come up tails. The probability doesn't change. It is the same thing for picking a nonce. Could the first nonce I plug in produce a hash that is below the difficulty level? Yes. And it is exactly the same [for] the second nonce I try. I could try them randomly or in decreasing order. I could start from any value of nonce, then add one and keep going. It doesn't matter. It makes no difference where you choose to start, or how you move from nonce to nonce. However, choosing randomly involves some more computation. You need to generate randomness; there is no reason to do that, because there is plenty of randomness in cryptographic hashes. Instead, just increment them by one. That is the fastest way to do it. Incrementing by one in binary is something that computers can do very quickly. Just pick a starting point for your nonce, increment by one, and you have the same chance... of finding a valid nonce as everybody else on the network.

How does a pool work with a lower difficulty level for its nodes to solve? If my node finds a nonce that produces a hash that is lower than the expected difficulty set by the pool, how does that relate to the difficulty set on the Bitcoin network?

Here is an interesting question. This is difficult to understand, but it is a very important concept, on how pools work with lower difficulty. "How does a pool work with a lower difficulty level for its nodes to solve?" "If my node finds a nonce that produces a hash that is lower than the expected difficulty set by the pool, how does that relate to the difficulty set on the Bitcoin network?" This is a really great question and needs more discussion to answer, so you can... grasp what is happening, but it is a neat trick. Let's say you have a room full of people who each hold two dice. You have a team working together, with the goal to throw dice below a certain number. If they [throw] below that certain number, they win. [Think of] that low number for the game the difficulty target for the blockchain. The players can throw dice between two and twelve; you can get the lowest (two 1s) or the highest (two 6s). We win if we roll less than five total between the two dice, so rolling a six won't work. Six and five doesn't work, five and five doesn't work, five and four doesn't work, four and three doesn't work, etc. Two and two works, one and two works, one and one works. Any of those throws. Below five is the target for the whole game. Your team starts throwing dice, and you want to reward the people who are doing the most throwing, because they are doing more work for the team. How do you know whether they are throwing dice often enough? You could just wait until one of them wins. But if you wait until one of them wins and give them [the whole reward], then it is not really a pool. That is the same as if they were competing by themselves. [Instead, you create] a pool strategy that gives a portion of the reward [to each team member], no matter who in the team throws [the winning dice]; the whole team wins. But you should split those winnings based on how much everybody contributed to the dice throwing. How would you do that? You set a difficulty target. Every time someone throws dice less than ten, they will [receive] one share of the team's contribution. So the team members throw dice. Twelve, eleven, and ten don't count. Nine is below the team's difficulty, though it is not below the game's difficulty. Because I threw a nine, I count one share and keep throwing, because the team difficulty is below ten. That will give us a pretty good metric as to who has done the most work. They will tend to throw more dice that is nine and below, so they will earn more shares. We will allocate shares based on throwing low numbers, but the probability [of hitting the target] is still the same. A winning dice throw below five is also below nine. The chances of throwing below nine are [greater] than throwing below five. They are not losing anything by announcing [scores]. "We didn't win the game, but I did win the pool [target]." That [strategy] will still help you get closer to winning the game, and it gives you a way to track performance. By setting these two difficulty levels, [the network difficulty versus the pool difficulty], you have the [target] at which we win the game, and the target at which you are recognized... as a team member for your work, though that is a less difficult throw. Everybody will start throwing their dice, trying to win the game as a pool, and then allocating winnings correctly... by counting the shares [for each team member]. That is how the pool difficulty works.

If I have an ASIC miner and I win the block reward, how will it be technically linked to a wallet of mine?

I hope that was understood. "If I have an ASIC miner and I [win the block] reward, how will it be technically linked to a wallet of mine?" "Can you explain the process I should follow? That is a great question, Scott. How does that work know who or where to pay when you win a block? It is quite simple, when you think about it. When you you construct a candidate block, the block you try to find proof-of-work for, that candidate block is made by your own computer. Your computer, connected to the ASIC miner, is collecting transactions and blocks from the network. It will construct a candidate block by taking a whole bunch of unconfirmed transactions, stuff them into a block, and then propose it as a new block to the rest of the network. You won't show this [candidate block] to anyone until you have proof-of-work, because no one is interested in a block that doesn't have proof-of-work. Within that block, there is a special transaction called the coinbase, which pays out the mining reward. Who creates that transaction? Well, your computer creates it. When you are a miner, your computer creates the candidate block and the coinbase transaction, which is currently a reward of 12.5 bitcoin plus fees from all the transactions in there. Therefore, you decide where that transaction pays to. Your computer [will insert] your destination address... into that candidate block while you are mining. If you are part of a pool, the pool operator does that. If you are mining by yourself, which you would only do if you have a lot of mining equipment, your computers will construct candidate blocks with a coinbase transaction that pays you. You write that transaction. Effectively, miners are writing cheques to themselves to collect the reward. A key part to understand: that cheque isn't valid unless the block is valid and there is proof-of-work. When a miner [produces] a proof-of-work solution for a block, then they write a cheque to themselves... for 12.5 bitcoin plus fees in the coinbase, which is a valid transaction. If that block is accepted by the rest of the network, it will show clearly in the blockchain that this miner... is being paid their reward. Of course, the reward needs to be the correct amount, the block subsidy plus fees. Everything in the block must be valid. Otherwise, it will rejected. If it is valid, there you go. The miner paid themselves [in the coinbase], [included] it in the candidate block, found the proof-of-work, and then telling everybody else so that block [will be included] in the blockchain. One hundred blocks later, the miner can spend [the reward from] that coinbase transaction. Therre is a one hundred block [waiting period] for coinbase transactions, where they can't be spent... until a hundred blocks have passed; in other words, coinbase transactions need one hundred confirmations. This rule is to prevent miners from using chain reorganizations to double-spend their own reward. "When a miner creates a block, am I right that this block is transmitted to all other nodes on the network, available for any miner to validate?" Well, not exactly. It is transmitted, but only after the proof-of-work is found. A miner only transmits a block once that miner has already proven it by finding a nonce for proof-of-work... within the difficulty target. The miner must [generate] quadrillions of hashes in order to find a suitable nonce. Only once that block contains a correct proof-of-work can the miner send it to the rest of the network. When the miner [broadcasts] that block to the first node, the node needs to validate the block's proof-of-work. If it doesn't have proof-of-work, the node will reject it and even disconnect from that miner. Every node on the network validates every transaction and block; miners cannot propagate blocks until those blocks are valid. They must have valid proof-of-work.