Repository: ethereum/consensus-specs

Issue:
Miscellaneous beacon chain changes—take 4
(See #128, #218, #322 for takes 1, 2, 3.)

Below is a list of miscellaneous suggestions for phase 0, most of which were discussed on the researcher's call on Feb 19. This issue keeps track of some of the phase 0 work remaining.

- [x] 1. **Friendlier GENESIS_SLOT**: Implemented in #655.
- [x] 2. **Granular state roots**: Expose state roots at every slot. Implemented in #649.
- [x] 3. **Previous block root reconstruction**: Provide enough information in `state` to reconstruct the current block's `previous_block_root`. Implemented in #649.
- [x] 4. **Define genesis Eth1 data**: Implemented in #649.
- [x] 5. **Mandatory deposits**: Mandatory processing of pending deposits.
- [x] 6. **Transfers during pre-activation**: Allow not-yet-activated validators to make transfers.
- [x] 7. **LMD GHOST tie breaker**: Compare block hashes to tie-break LMD GHOST.
- [ ] 8. **Maximum reversions**: Enshrine dynamic weak subjectivity revert period. See #577.
- [x] 9. **Double justifications**: Specify fork choice rule when there are two justified blocks at the same height. (Possible solution: ignore both and fallback to the previous highest justified block.)
- [x] 10. **Move to SHA256**: See #612.
- [ ] 11. **Standardise BLS12-381**: See #605.
- [ ] 12. **Performance parameters tuning**: Fine-tune `SECONDS_PER_SLOT`, `SHARD_COUNT`, etc. based on benchmarks.
- [ ] 13a. **Increase proposer rewards**: See #621. Need to check incentive compatibility with inclusion distance reward.
- [x] 13b. **Incentive-compatible proposer rewards**: Make proposer rewards proportional to balance. 
- [ ] 14. **Increase rewards in general**: Calculate theoretical max issuance rate and work backwards to expected issuance.
- [x] 15. **Reduce SSZ_CHUNK_SIZE to 32**: See #603 and #696.
- [x] 16. **SSZ tuples**: See #665 and #696.
- [x] 
17. **Immediately withdrawable if bad proof of possession**: See #657.

- [x] 18. **4-byte working balance**: See #685.
- [x] 19. **Merkleisation-friendly pending attestations**: See #697.
- [ ] 20. **Fine-tune container field ordering**: To do with constants fine-tuning.
- [x] 21. **Minimum activation period**: See [here](https://github.com/ethereum/eth2.0-specs/issues/675#issuecomment-468159678) and [here](https://github.com/ethereum/eth2.0-specs/pull/746).
- [x] 22. **Milder ejections**: Replace `exit_validator` by `initiate_validator_exit` in `process_ejections`.
- [x] 23. **Improved rate limiting**: Change the rate limiting logic (for entry/exit/withdrawal) based on [this Ethresear.ch post](https://ethresear.ch/t/rate-limiting-entry-exits-not-withdrawals/4942).
- [x] 24. **Epoch transitions at start of epoch**: Instead of at the very end of the epoch.
- [x] 25. **Epoch-based proposer slashing**: As opposed to slot-based.
- [x] 26. **Genesis epochs**: Use `GENESIS_EPOCH - 1` for `previous_shuffling_epoch` and maybe `previous_shuffling_epoch`.
- [x] 
27. **No backfilling of latest_active_index_roots**: Only set the active index root for the first slot.

- [x] 28. 
**`start_shard` offsets**: For fairer crosslinking latency across shards.

- [x] 29. **Remove deposit timestamps and `DepositData`**: See #760.
- [x] 30. **Fair proposer sampling**: See #733.
- [ ] 31. **Slashed validators and LMD GHOST**: Should attestations from slashed validators be ignored in LMD GHOST?
- [x] 32. **Incentives simplification**: Simplification of the rewards and penalties.
- [ ] 33. **Exit fee**: See [here](https://github.com/ethereum/eth2.0-specs/pull/850#issuecomment-478068655).
- [x] 34. **GENESIS_SLOT == 0**: From Danny.
- [ ] 35. **Incentive-compatible crosslink rewards**: Proportional to amount of crosslink data.
- [ ] 36. **No phase 0 transfers**: Push transfers to phase 1 so that no economically meaningful activity happens during phase 0. This allows for phase 0 (a "testnet") to be rebooted if things go horribly wrong.
- [ ] 37. **Explicit genesis deposits**: Put genesis deposits in `block.body.deposits`.
- [x] 38. **Remove serialization from consensus**: See #924.
- [ ] 39. **Do not store withdrawal credentials**: See #937.
- [ ] 40. **Increase SECONDS_PER_SLOT and remove MIN_ATTESTATION_INCLUSION_DELAY**: The idea is to set different `SECONDS_PER_BEACON_SLOT`and `SECONDS_PER_SHARD_SLOT`, e.g. to 8/4, 12/3 or 16/4.
- [ ] 41. **The slotering**: Remove various unnecessary slots and replace by epochs where appropriate. (Justin surprise cleanup.)
- [x] 42. **Graffiti**: 32-byte arbitrary data in blocks
- [ ] 43. **Merge historical stats**: In particular, merge constants under "State list lengths".
- [ ] 44. **Improve epoch processing**: See #1043.


Discussion:
>  17. Immediately withdrawable if bad proof of possession: See #657.

We decided during the call to not do that, correct?

>  13. Increase proposer rewards: See #621. Need to check incentive compatibility with inclusion distance reward.

If we are going to do this, then making proposer rewards proportional to balance should arguably be in there as well. But it's not yet clear to be that we should do this; as I mentioned in the call, proposer rewards are already an expected 1/9 of validator's total revenue, and expected to increase in phase 1 due to minor rewards from the various custody objects.
Also, did we want to add a mandatory minimum time that a validator must have been deposited before they can exit? This would (i) ensure stability of persistent committees and (ii) help put a cap on validator registry modification events.
>  23. Improved rate limiting: Change the rate limiting logic (for entry/exit/withdrawal) based on this Ethresear.ch post.

I asked econoar to study more to make sure this is a good idea. 
>13a and 13b

#735
>  27. No backfilling of latest_active_index_roots: Only set the active index root for the first slot.

Why? In the [light client spec](https://notes.ethereum.org/s/B1DtMJZeV#), I use the `latest_active_index_roots` as a way of getting the historical proposal committee, and removing the backfilling would require me to add extra special case logic there for the near-genesis case.
>  9. Double justifications: Specify fork choice rule when there are two justified blocks at the same height. (Possible solution: ignore both and fallback to the previous highest justified block.)

I'd say we just use the same rule we do in other contexts and sort lexicographically by hash.

Resolving patch:
diff --git a/utils/phase0/state_transition.py b/utils/phase0/state_transition.py
--- a/utils/phase0/state_transition.py
+++ b/utils/phase0/state_transition.py
@@ -15,6 +15,13 @@
 )
 
 
+def expected_deposit_count(state: BeaconState) -> int:
+    return min(
+        spec.MAX_DEPOSITS,
+        state.latest_eth1_data.deposit_count - state.deposit_index
+    )
+
+
 def process_transaction_type(state: BeaconState,
                              transactions: List[Any],
                              max_transactions: int,
@@ -31,30 +38,36 @@ def process_transactions(state: BeaconState, block: BeaconBlock) -> None:
         spec.MAX_PROPOSER_SLASHINGS,
         spec.process_proposer_slashing,
     )
+
     process_transaction_type(
         state,
         block.body.attester_slashings,
         spec.MAX_ATTESTER_SLASHINGS,
         spec.process_attester_slashing,
     )
+
     process_transaction_type(
         state,
         block.body.attestations,
         spec.MAX_ATTESTATIONS,
         spec.process_attestation,
     )
+
+    assert len(block.body.deposits) == expected_deposit_count(state)
     process_transaction_type(
         state,
         block.body.deposits,
         spec.MAX_DEPOSITS,
         spec.process_deposit,
     )
+
     process_transaction_type(
         state,
         block.body.voluntary_exits,
         spec.MAX_VOLUNTARY_EXITS,
         spec.process_voluntary_exit,
     )
+
     assert len(block.body.transfers) == len(set(block.body.transfers))
     process_transaction_type(
         state,


Test patch:
diff --git a/tests/phase0/block_processing/test_process_deposit.py b/tests/phase0/block_processing/test_process_deposit.py
new file mode 100644
--- /dev/null
+++ b/tests/phase0/block_processing/test_process_deposit.py
@@ -0,0 +1,132 @@
+from copy import deepcopy
+import pytest
+
+import build.phase0.spec as spec
+
+from build.phase0.spec import (
+    Deposit,
+    process_deposit,
+)
+from tests.phase0.helpers import (
+    build_deposit,
+)
+
+
+# mark entire file as 'voluntary_exits'
+pytestmark = pytest.mark.voluntary_exits
+
+
+def test_success(state, deposit_data_leaves, pubkeys, privkeys):
+    pre_state = deepcopy(state)
+
+    index = len(deposit_data_leaves)
+    pubkey = pubkeys[index]
+    privkey = privkeys[index]
+    deposit, root, deposit_data_leaves = build_deposit(
+        pre_state,
+        deposit_data_leaves,
+        pubkey,
+        privkey,
+        spec.MAX_DEPOSIT_AMOUNT,
+    )
+
+    pre_state.latest_eth1_data.deposit_root = root
+    pre_state.latest_eth1_data.deposit_count = len(deposit_data_leaves)
+
+    post_state = deepcopy(pre_state)
+
+    process_deposit(post_state, deposit)
+
+    assert len(post_state.validator_registry) == len(state.validator_registry) + 1
+    assert len(post_state.validator_balances) == len(state.validator_balances) + 1
+    assert post_state.validator_registry[index].pubkey == pubkeys[index]
+    assert post_state.deposit_index == post_state.latest_eth1_data.deposit_count
+
+    return pre_state, deposit, post_state
+
+
+def test_success_top_up(state, deposit_data_leaves, pubkeys, privkeys):
+    pre_state = deepcopy(state)
+
+    validator_index = 0
+    amount = spec.MAX_DEPOSIT_AMOUNT // 4
+    pubkey = pubkeys[validator_index]
+    privkey = privkeys[validator_index]
+    deposit, root, deposit_data_leaves = build_deposit(
+        pre_state,
+        deposit_data_leaves,
+        pubkey,
+        privkey,
+        amount,
+    )
+
+    pre_state.latest_eth1_data.deposit_root = root
+    pre_state.latest_eth1_data.deposit_count = len(deposit_data_leaves)
+    pre_balance = pre_state.validator_balances[validator_index]
+
+    post_state = deepcopy(pre_state)
+
+    process_deposit(post_state, deposit)
+
+    assert len(post_state.validator_registry) == len(state.validator_registry)
+    assert len(post_state.validator_balances) == len(state.validator_balances)
+    assert post_state.deposit_index == post_state.latest_eth1_data.deposit_count
+    assert post_state.validator_balances[validator_index] == pre_balance + amount
+
+    return pre_state, deposit, post_state
+
+
+def test_wrong_index(state, deposit_data_leaves, pubkeys, privkeys):
+    pre_state = deepcopy(state)
+
+    index = len(deposit_data_leaves)
+    pubkey = pubkeys[index]
+    privkey = privkeys[index]
+    deposit, root, deposit_data_leaves = build_deposit(
+        pre_state,
+        deposit_data_leaves,
+        pubkey,
+        privkey,
+        spec.MAX_DEPOSIT_AMOUNT,
+    )
+
+    # mess up deposit_index
+    deposit.index = pre_state.deposit_index + 1
+
+    pre_state.latest_eth1_data.deposit_root = root
+    pre_state.latest_eth1_data.deposit_count = len(deposit_data_leaves)
+
+    post_state = deepcopy(pre_state)
+
+    with pytest.raises(AssertionError):
+        process_deposit(post_state, deposit)
+
+    return pre_state, deposit, None
+
+
+def test_bad_merkle_proof(state, deposit_data_leaves, pubkeys, privkeys):
+    pre_state = deepcopy(state)
+
+    index = len(deposit_data_leaves)
+    pubkey = pubkeys[index]
+    privkey = privkeys[index]
+    deposit, root, deposit_data_leaves = build_deposit(
+        pre_state,
+        deposit_data_leaves,
+        pubkey,
+        privkey,
+        spec.MAX_DEPOSIT_AMOUNT,
+    )
+
+    # mess up merkle branch
+    deposit.proof[-1] = spec.ZERO_HASH
+
+    pre_state.latest_eth1_data.deposit_root = root
+    pre_state.latest_eth1_data.deposit_count = len(deposit_data_leaves)
+
+    post_state = deepcopy(pre_state)
+
+    with pytest.raises(AssertionError):
+        process_deposit(post_state, deposit)
+
+    return pre_state, deposit, None
diff --git a/tests/phase0/block_processing/test_voluntary_exit.py b/tests/phase0/block_processing/test_voluntary_exit.py
--- a/tests/phase0/block_processing/test_voluntary_exit.py
+++ b/tests/phase0/block_processing/test_voluntary_exit.py
@@ -13,6 +13,10 @@
 )
 
 
+# mark entire file as 'voluntary_exits'
+pytestmark = pytest.mark.voluntary_exits
+
+
 def test_success(state, pub_to_priv):
     pre_state = deepcopy(state)
     #
diff --git a/tests/phase0/helpers.py b/tests/phase0/helpers.py
--- a/tests/phase0/helpers.py
+++ b/tests/phase0/helpers.py
@@ -72,12 +72,16 @@ def create_mock_genesis_validator_deposits(num_validators, deposit_data_leaves):
 
 
 def create_genesis_state(num_validators, deposit_data_leaves):
-    initial_deposits, deposit_root = create_mock_genesis_validator_deposits(num_validators, deposit_data_leaves)
+    initial_deposits, deposit_root = create_mock_genesis_validator_deposits(
+        num_validators,
+        deposit_data_leaves,
+    )
     return get_genesis_beacon_state(
         initial_deposits,
         genesis_time=0,
         genesis_eth1_data=Eth1Data(
             deposit_root=deposit_root,
+            deposit_count=len(initial_deposits),
             block_hash=spec.ZERO_HASH,
         ),
     )
@@ -171,3 +175,27 @@ def build_voluntary_exit(state, epoch, validator_index, privkey):
     )
 
     return voluntary_exit
+
+
+def build_deposit(state,
+                  deposit_data_leaves,
+                  pubkey,
+                  privkey,
+                  amount):
+    deposit_data = build_deposit_data(state, pubkey, privkey, amount)
+
+    item = hash(deposit_data.serialize())
+    index = len(deposit_data_leaves)
+    deposit_data_leaves.append(item)
+    tree = calc_merkle_tree_from_leaves(tuple(deposit_data_leaves))
+    root = get_merkle_root((tuple(deposit_data_leaves)))
+    proof = list(get_merkle_proof(tree, item_index=index))
+    assert verify_merkle_branch(item, proof, spec.DEPOSIT_CONTRACT_TREE_DEPTH, index, root)
+
+    deposit = Deposit(
+        proof=list(proof),
+        index=index,
+        deposit_data=deposit_data,
+    )
+
+    return deposit, root, deposit_data_leaves
diff --git a/tests/phase0/test_sanity.py b/tests/phase0/test_sanity.py
--- a/tests/phase0/test_sanity.py
+++ b/tests/phase0/test_sanity.py
@@ -196,6 +196,7 @@ def test_deposit_in_block(state, deposit_data_leaves, pubkeys, privkeys):
     )
 
     pre_state.latest_eth1_data.deposit_root = root
+    pre_state.latest_eth1_data.deposit_count = len(test_deposit_data_leaves)
     post_state = deepcopy(pre_state)
     block = build_empty_block_for_next_slot(post_state)
     block.body.deposits.append(deposit)
@@ -233,6 +234,7 @@ def test_deposit_top_up(state, pubkeys, privkeys, deposit_data_leaves):
     )
 
     pre_state.latest_eth1_data.deposit_root = root
+    pre_state.latest_eth1_data.deposit_count = len(test_deposit_data_leaves)
     block = build_empty_block_for_next_slot(pre_state)
     block.body.deposits.append(deposit)