Increasing the blocksize as a (generalized) softfork.

Increasing the blocksize as a (generalized) softfork.
2015-12-20 15:22 ` joe2015
                   ` (4 more replies)
  
From: joe2015 @ 2015-12-20 10:56 UTC (permalink / raw)
  To: bitcoin-dev

This is a draft.

--joe

Introduction
============

It is generally assumed that increasing the blocksize limit requires a
hardfork.  Instead we show that a increasing the limit can be achieved 
using a
"generalized" softfork.  Like standard softforks, generalized softforks 
need a
mere miner majority (>50% hashpower) rather than global consensus.

Standard Softforks
==================

After a softfork two potential chains exist:

* The new chain B3,B4,B5,... valid under the new rules and old rules.
* The old chain B3',B4',B5',... valid under the old rules only.

E.g.

                       +-- B3 --- B4 --- B5
                       |
     ... -- B1 -- B2 --+
                       |
                       +-- B3' -- B4' -- B5' -- B6'

Assuming that >50% of the hashpower follow the new rules, the old chain 
is
doomed to be orphaned:

                       +-- B3 --- B4 --- B5 --- B6 --- B7 --- B8 --- ...
                       |
     ... -- B1 -- B2 --+
                       |
                       +-- B3' -- B4' -- B5' -- B6' (orphaned)

Hardforks may result in two chains that can co-exist indefinitely:

                       +-- B3 --- B4 --- B5 --- B6 --- B7 --- B8 --- ...
                       |
     ... -- B1 -- B2 --+
                       |
                       +-- B3' -- B4' -- B5' -- B6' -- B7' -- B8' -- ...

Generalized Softforks
=====================

A *generalized* softfork introduces a transform function f(B)=B' that 
maps a
block B valid under the new rules to a block B' valid under the old 
rules.

After a generalized softfork three chains may exist:

* The new chain B3,B4,B5,... valid under the new rules only.
* The mapped chain f(B3),f(B4),f(B5),... valid under the old rules.
* The old chain B3',B4',B5',... valid under the old rules only.

E.g.

                       +-- B3 ---- B4 ---- B5
                       |
     ... -- B1 -- B2 --+-- f(B3) - f(B4) - f(B5)
                       |
                       +-- B3' --- B4' --- B5' --- B6'

This is "generalized" softfork since defining f(B)=B (identity function)
reduces to the standard softfork case above.

As with standard softforks, if the majority of the hashpower follow the 
new
rules then the old chain B3',B4',B5',... is doomed to be orphaned:

                       +-- B3 ---- B4 ---- B5 ---- B6 ---- B7 ---- ...
                       |
     ... -- B1 -- B2 --+-- f(B3) - f(B4) - f(B5) - f(B6) - f(B7) - ...
                       |
                       +-- B3' --- B4' --- B5' --- B6' (orphaned)

Example:
--------

Segregated Witness can be thought of as an example of a generalized 
softfork.
Here the new block format consists of the combined old block and witness 
data.
The function f() simply strips the witness data to reveal a valid block 
under
the old rules:

     NewBlock := OldBlock ++ Witness
     f(NewBlock) = OldBlock

An Arbitrary Block-size Increase Via a Generalized Softfork
===========================================================

Segregated Witness only allows for a modest effective blocksize increase
(although there can be other motivations for SW, but that is off-topic).

Instead we engineer a generalized softfork that allows an arbitrarily 
increase
of the blocksize limit.  The proposal consists of two parts: (a) new 
rules for
valid blocks, and (b) a transformation function f().

The new block rules are very similar to the old block rules but with 
some
small changes.  In summary the changes are:

* The MAX_BLOCK_SIZE limit is raised to some new limit
   (e.g. 8Mb, BIP101, 2-4-8, BIP202, etc., or some other limit)
* The MerkleRoot field in the header has been re-interpreted.
* The CoinBaseTx must obey some additional new rules.

As with old blocks, a block under the new rules consists of a block 
header
followed by a vector of transactions [CoinBaseTx, Tx1, .., Txn], i.e.

     NewBlock := BlockHeader ++ NumTx ++ CoinBaseTx ++ Tx1 ++ .. ++ Txn

The block header format is the same as under the old rules defined as 
follows:

     
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |  Ver  |                        PrevHash                            
    |
     
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                           MerkleRoot                          | 
Time  |
     
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     | Bits  | Nonce |
     +-+-+-+-+-+-+-+-+

Under the old rules MerkleRoot is the Merkle root of the hashes of all
transactions included in the block, i.e.

     MerkleRoot = merkleRoot([hash(CoinBaseTx), hash(Tx1), .., 
hash(Txn)])

Under the new rules we instead define:

     MerkleRoot = merkleRoot([hash(CoinBaseTx)])

That is, under the new rules, MerkleRoot is the Merkle root of a 
singleton
vector containing the CoinBaseTx hash only.

In order to preserve the security properties of Bitcoin we additionally
require that the CoinBaseTx somehow encodes the Merkle root of the 
remaining
transactions [Tx1, .., Txn].  For example, this could be achieved by 
requiring
a mandatory OP_RETURN output that encodes this information, e.g.

     OP_RETURN merkleRoot([hash(Tx1), .., hash(Txn)])

Alternatively the Merkle root could be encoded in the coinbase itself.  
This
ensures that new transactions cannot be added/deleted from the block 
without
altering the MerkleRoot field in the header.

Aside from these changes and the increased MAX_BLOCK_SIZE, the new block 
must
obey all the rules of the old block format, e.g. valid PoW, have valid 
block
reward, contain valid transactions, etc., etc.

In order to be a generalized softfork we also need to define a mapping 
f()
from valid new blocks to valid blocks under the old rules.  We can 
define this
as follows:

     NewBlock    := BlockHeader ++ NumTx ++ CoinBaseTx ++ Tx1 ++ .. ++ 
Txn
     f(NewBlock) := BlockHeader ++ 1 ++ CoinBaseTx

That is, function f() simply truncates the block so that it contains the
coinbase transaction only.  After truncation, the MerkleRoot field of 
the
block header is valid under the old rules.

The proposed new rules combined with the transformation f() comprise a
generalized softfork.  After the fork a new chain B3,B4,B5,... will be
generated under the new rules defined above, including an increased 
blocksize
limit.  This new chain can be mapped to a valid chain 
f(B3),f(B4),f(B5),...
under the old rules.  Assuming that >50% of the hashpower has adopted 
the new
rules, the mapped chain will orphan any competing chain under the old 
rules,
just like a standard softfork.

An interesting consequence of this design is that, since all mapped 
blocks are
empty, old clients will never see transactions confirming.  This is be a
strong incentive for users to update their clients.

Conclusion
==========

Conventional wisdoms suggests that increasing the blocksize limit 
requires a
hardfork.  We show that it can instead be achieved using a generalized
softfork.  Like with a standard softfork, a generalized softfork merely
requires a majority (>50%) of hash power rather than global consensus.
Experience has shown that the former is significantly easier to achieve.

Future Work
-----------

Investigate other kinds of hardforks that can instead be implemented as
generalized softforks, and the security implications of such...

7943a2934d0be2f96589fdef2b2e00a2a7d8c3b782546bb37625d1669accb9b1
72f018588572ca2786168cb531d10e79b81b86d3fada92298225a0f950eed3a5



 permalink raw reply	[|nested]
2015-12-20 15:22 ` 
                   ` 
  
From: joe2015 @ 2015-12-20 10:56 UTC (permalink / raw)
  To: bitcoin-dev

This is a draft.

--joe

Introduction
============

It is generally assumed that increasing the blocksize limit requires a
hardfork.  Instead we show that a increasing the limit can be achieved 
using a
"generalized" softfork.  Like standard softforks, generalized softforks 
need a
mere miner majority (>50% hashpower) rather than global consensus.

Standard Softforks
==================

After a softfork two potential chains exist:

* The new chain B3,B4,B5,... valid under the new rules and old rules.
* The old chain B3',B4',B5',... valid under the old rules only.

E.g.

                       +-- B3 --- B4 --- B5
                       |
     ... -- B1 -- B2 --+
                       |
                       +-- B3' -- B4' -- B5' -- B6'

Assuming that >50% of the hashpower follow the new rules, the old chain 
is
doomed to be orphaned:

                       +-- B3 --- B4 --- B5 --- B6 --- B7 --- B8 --- ...
                       |
     ... -- B1 -- B2 --+
                       |
                       +-- B3' -- B4' -- B5' -- B6' (orphaned)

Hardforks may result in two chains that can co-exist indefinitely:

                       +-- B3 --- B4 --- B5 --- B6 --- B7 --- B8 --- ...
                       |
     ... -- B1 -- B2 --+
                       |
                       +-- B3' -- B4' -- B5' -- B6' -- B7' -- B8' -- ...

Generalized Softforks
=====================

A *generalized* softfork introduces a transform function f(B)=B' that 
maps a
block B valid under the new rules to a block B' valid under the old 
rules.

After a generalized softfork three chains may exist:

* The new chain B3,B4,B5,... valid under the new rules only.
* The mapped chain f(B3),f(B4),f(B5),... valid under the old rules.
* The old chain B3',B4',B5',... valid under the old rules only.

E.g.

                       +-- B3 ---- B4 ---- B5
                       |
     ... -- B1 -- B2 --+-- f(B3) - f(B4) - f(B5)
                       |
                       +-- B3' --- B4' --- B5' --- B6'

This is "generalized" softfork since defining f(B)=B (identity function)
reduces to the standard softfork case above.

As with standard softforks, if the majority of the hashpower follow the 
new
rules then the old chain B3',B4',B5',... is doomed to be orphaned:

                       +-- B3 ---- B4 ---- B5 ---- B6 ---- B7 ---- ...
                       |
     ... -- B1 -- B2 --+-- f(B3) - f(B4) - f(B5) - f(B6) - f(B7) - ...
                       |
                       +-- B3' --- B4' --- B5' --- B6' (orphaned)

Example:
--------

Segregated Witness can be thought of as an example of a generalized 
softfork.
Here the new block format consists of the combined old block and witness 
data.
The function f() simply strips the witness data to reveal a valid block 
under
the old rules:

     NewBlock := OldBlock ++ Witness
     f(NewBlock) = OldBlock

An Arbitrary Block-size Increase Via a Generalized Softfork
===========================================================

Segregated Witness only allows for a modest effective blocksize increase
(although there can be other motivations for SW, but that is off-topic).

Instead we engineer a generalized softfork that allows an arbitrarily 
increase
of the blocksize limit.  The proposal consists of two parts: (a) new 
rules for
valid blocks, and (b) a transformation function f().

The new block rules are very similar to the old block rules but with 
some
small changes.  In summary the changes are:

* The MAX_BLOCK_SIZE limit is raised to some new limit
   (e.g. 8Mb, BIP101, 2-4-8, BIP202, etc., or some other limit)
* The MerkleRoot field in the header has been re-interpreted.
* The CoinBaseTx must obey some additional new rules.

As with old blocks, a block under the new rules consists of a block 
header
followed by a vector of transactions [CoinBaseTx, Tx1, .., Txn], i.e.

     NewBlock := BlockHeader ++ NumTx ++ CoinBaseTx ++ Tx1 ++ .. ++ Txn

The block header format is the same as under the old rules defined as 
follows:

     
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |  Ver  |                        PrevHash                            
    |
     
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                           MerkleRoot                          | 
Time  |
     
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     | Bits  | Nonce |
     +-+-+-+-+-+-+-+-+

Under the old rules MerkleRoot is the Merkle root of the hashes of all
transactions included in the block, i.e.

     MerkleRoot = merkleRoot([hash(CoinBaseTx), hash(Tx1), .., 
hash(Txn)])

Under the new rules we instead define:

     MerkleRoot = merkleRoot([hash(CoinBaseTx)])

That is, under the new rules, MerkleRoot is the Merkle root of a 
singleton
vector containing the CoinBaseTx hash only.

In order to preserve the security properties of Bitcoin we additionally
require that the CoinBaseTx somehow encodes the Merkle root of the 
remaining
transactions [Tx1, .., Txn].  For example, this could be achieved by 
requiring
a mandatory OP_RETURN output that encodes this information, e.g.

     OP_RETURN merkleRoot([hash(Tx1), .., hash(Txn)])

Alternatively the Merkle root could be encoded in the coinbase itself.  
This
ensures that new transactions cannot be added/deleted from the block 
without
altering the MerkleRoot field in the header.

Aside from these changes and the increased MAX_BLOCK_SIZE, the new block 
must
obey all the rules of the old block format, e.g. valid PoW, have valid 
block
reward, contain valid transactions, etc., etc.

In order to be a generalized softfork we also need to define a mapping 
f()
from valid new blocks to valid blocks under the old rules.  We can 
define this
as follows:

     NewBlock    := BlockHeader ++ NumTx ++ CoinBaseTx ++ Tx1 ++ .. ++ 
Txn
     f(NewBlock) := BlockHeader ++ 1 ++ CoinBaseTx

That is, function f() simply truncates the block so that it contains the
coinbase transaction only.  After truncation, the MerkleRoot field of 
the
block header is valid under the old rules.

The proposed new rules combined with the transformation f() comprise a
generalized softfork.  After the fork a new chain B3,B4,B5,... will be
generated under the new rules defined above, including an increased 
blocksize
limit.  This new chain can be mapped to a valid chain 
f(B3),f(B4),f(B5),...
under the old rules.  Assuming that >50% of the hashpower has adopted 
the new
rules, the mapped chain will orphan any competing chain under the old 
rules,
just like a standard softfork.

An interesting consequence of this design is that, since all mapped 
blocks are
empty, old clients will never see transactions confirming.  This is be a
strong incentive for users to update their clients.

Conclusion
==========

Conventional wisdoms suggests that increasing the blocksize limit 
requires a
hardfork.  We show that it can instead be achieved using a generalized
softfork.  Like with a standard softfork, a generalized softfork merely
requires a majority (>50%) of hash power rather than global consensus.
Experience has shown that the former is significantly easier to achieve.

Future Work
-----------

Investigate other kinds of hardforks that can instead be implemented as
generalized softforks, and the security implications of such...

7943a2934d0be2f96589fdef2b2e00a2a7d8c3b782546bb37625d1669accb9b1
72f018588572ca2786168cb531d10e79b81b86d3fada92298225a0f950eed3a5



 permalink raw reply	[|nested]
* 
  2015-12-20 10:56 [bitcoin-dev] Increasing the blocksize as a (generalized) softfork joe2015

  2015-12-20 15:50 ` 
                   ` 
  4 siblings, 0 replies; 18+ messages in thread
From: joe2015 @ 2015-12-20 15:22 UTC (permalink / raw)
  To: bitcoin-dev

Link to better formatted version for web-users:
https://bitcointalk.org/index.php?topic=1296628.0

--joe


^ permalink raw reply	[|nested] 18+ messages in thread
2015-12-21  3:39 ` Jeff Garzik
  
From: joe2015 @ 2015-12-20 17:21 UTC (permalink / raw)
  To: bitcoin-dev

On 2015-12-20 23:50, Tier Nolan via bitcoin-dev wrote:
> This is essentially the "nuclear option".

Remember this is proposed as an alternative to hardforks, which is also 
a "nuclear option".  Hardforks carry significant risks such as 
permanently splitting Bitcoin into two chains if global consensus is 
never reached.  A (generalized) softfork avoids this problem.

> This could be achieved by adding the hash of an extended block into
> the coinbase but not requiring the coinbase to be the only
> transaction.

I think this can also be viewed as a generalized softfork if one so 
chooses, e.g.

     NewBlock := OldBlock ++ ExtendedBlock
     f(NewBlock) = OldBlock

I do not think this is a bad idea but is more complex than my proposal, 
e.g. users having to move coins between different tiers of blocks.  
Under my proposal the Bitcoin works more or less the same except with a 
larger limit.

--joe


 permalink raw reply	[|nested]
* 
  2015-12-20 10:56 [bitcoin-dev] Increasing the blocksize as a (generalized) softfork joe2015
  2015-12-20 15:22 ` joe2015

  2015-12-20 18:17 ` 
                   ` 
  4 siblings, 0 replies; 18+ messages in thread
From: Tier Nolan @ 2015-12-20 15:50 UTC (permalink / raw)
  Cc: Bitcoin Dev

[-- Attachment #1: Type: text/plain, Size: 1762 bytes --]

This is essentially the "nuclear option".  You are destroying the current
chain (converting it to a chain of coinbases) and using the same POW to
start the new chain.  You are also giving everyone credit in the new chain
equal to their credit in the old chain.

It would be better if the current chain wasn't destroyed.

This could be achieved by adding the hash of an extended block into the
coinbase but not requiring the coinbase to be the only transaction.

The new block is the legacy block plus the associated extended block.

Users would be allowed to move money to the extended block by spending it
to a specific output template.

<public key hash> OP_1 OP_TO_EXTENDED OP_TRUE

OP_1 is the extended block index and initially, only one level is available.

This would work like P2SH.  Users could spend the money on the extended
block chain exactly as they could on the main chain.

Money can be brought back the same way.

<public key hash> <txid1> <txid2> ... <txid-n> <N> OP_0 OP_UNLOCK OP_TRUE

The txids are for transactions that have been locked in root chain.  The
transaction is only valid if they are all fully funded.  The fee for the
transaction would be fee - (cost to fund unlocked txids).  A negative fee
tx would be invalid.

This has the advantage that it keeps the main chain operating.  People can
still send money with their un-upgraded clients.  There is also an
incentive to move funds to the extended block(s).  The new extended blocks
are more complex, but potentially have lower fees.  Nobody is forced to
change.  If the large blocks aren't needed, nobody will both to use them.

The rule could be

Now:
0) 1 MB

After change over
0) 1 MB
1) 2 MB

After 2 years
0) 1 MB
1) 2 MB
2) 4MB

After 4 years
0) 1 MB
1) 2 MB
2) 4MB
3) 8MB

[-- Attachment #2: Type: text/html, Size: 2307 bytes --]

^ permalink raw reply	[|nested] 18+ messages in thread
2015-12-20 18:17 ` Bryan Bishop

  
From: joe2015 @ 2015-12-21  3:04 UTC (permalink / raw)
  To: Bryan Bishop;  Bitcoin Dev

On 2015-12-21 02:17, Bryan Bishop wrote:
> On Sun, Dec 20, 2015 at 4:56 AM, joe2015--- via bitcoin-dev
> <bitcoin-dev@lists•linuxfoundation.org> wrote:
> 
>> An Arbitrary Block-size Increase Via a Generalized Softfork
> 
> This seems conceptually similar to "extension blocks":
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-May/008356.html
> [1]
> https://bitcointalk.org/index.php?topic=283746.0 [2]
> http://gnusha.org/bitcoin-wizards/2015-12-20.log [3]
> 
> "Extended blocks" are also mentioned over here too:
> https://bitcointalk.org/index.php?topic=1296628.msg13307275#msg13307275
> [4]

The main difference is that my proposal does not introduce different 
"tiers" of blocks, and does not require uses to move coins to manually 
move coins between these tiers.

Instead, my proposal uses a single flat block format that is essentially 
the same as the current block format; only bigger.

The main point is that such a change does not require a hardfork with 
global consensus, as is commonly assumed, but rather can be deployed 
like a softfork using the method described in my original post.

--joe.


 permalink raw reply	[|nested]
2015-12-20 17:21 joe2015

  2015-12-21  3:58   ` joe2015
  
From: Jeff Garzik @ 2015-12-21  3:39 UTC (permalink / raw)
  To: joe2015;  Bitcoin development mailing list

[-- Attachment #1: Type: text/plain, Size: 543 bytes --]

On Sun, Dec 20, 2015 at 12:21 PM, joe2015--- via bitcoin-dev <
bitcoin-dev@lists•linuxfoundation.org> wrote:

> Remember this is proposed as an alternative to hardforks, which is also a
> "nuclear option".  Hardforks carry significant risks such as permanently
> splitting Bitcoin into two chains if global consensus is never reached.  A
> (generalized) softfork avoids this problem.


Current hard fork implementations include / will include miner lock-in,
just like any soft fork.  They will not activate if global consensus is not
reached.

[-- Attachment #2: Type: text/html, Size: 935 bytes --]

 permalink raw reply	[|nested]
2015-12-21  3:39 ` Jeff Garzik

  
From: joe2015 @ 2015-12-21  3:58 UTC (permalink / raw)
  To: Jeff Garzik;  Bitcoin development mailing list

On 2015-12-21 11:39, Jeff Garzik wrote:
> On Sun, Dec 20, 2015 at 12:21 PM, joe2015--- via bitcoin-dev
> <bitcoin-dev@lists•linuxfoundation.org> wrote:

> Current hard fork implementations include / will include miner
> lock-in, just like any soft fork.  They will not activate if global
> consensus is not reached.

That's not true at all. They activate with a miner majority (e.g. 75%, 
95%, etc.), not global consensus.  Here global really means global, i.e. 
miner, economic, all clients, etc.  In the case of a hardfork there is 
nothing stopping the miner minority from continuing the old chain.  With 
a softfork the miner minority is forced to upgrade otherwise their 
blocks will be eventually orphaned.

My proposal achieves a hardfork-like blocksize limit increase but, like 
a softfork, also forces the miner minority to upgrade.

--joe.


 permalink raw reply	[|nested]
2015-12-20 10:56 [bitcoin-dev] Increasing the blocksize as a (generalized) softfork joe2015
                   ` (2 preceding siblings ...)
  2015-12-20 18:17 ` Bryan Bishop

  2015-12-21  4:41   ` joe2015
  2016-01-04 21:53 ` Luke Dashjr
  
From: jl2012 @ 2015-12-21  4:23 UTC (permalink / raw)
  To: joe2015;  bitcoin-dev

I proposed something very similar 2 years ago:
https://bitcointalk.org/index.php?topic=283746.0

This is an interesting academic idea. But the way you implement it will 
immediately kill all existing full and SPV nodes (not really dead, 
rather like zombie as they can't send and receive any tx).

joe2015--- via bitcoin-dev 於 2015-12-20 05:56 寫到:
> This is a draft.
> 
> --joe
> 
> Introduction
> ============
> 
> It is generally assumed that increasing the blocksize limit requires a
> hardfork.  Instead we show that a increasing the limit can be achieved 
> using a
> "generalized" softfork.  Like standard softforks, generalized softforks 
> need a
> mere miner majority (>50% hashpower) rather than global consensus.
> 



 permalink raw reply	[|nested]
2015-12-21  4:23 ` jl2012

  2015-12-30 19:00     ` Bob McElrath
  
From: joe2015 @ 2015-12-21  4:41 UTC ( / )
  To: jl2012;  bitcoin-dev

On 2015-12-21 12:23, jl2012 wrote:
> I proposed something very similar 2 years ago:
> https://bitcointalk.org/index.php?topic=283746.0

Yes there are similarities but also some important differences.  See my 
response here: 
http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/012085.html

In short my proposal is compatible with conventional blocksize limit 
hardfork ideas, like BIP101, BIP202, 2-4-8 etc. etc.

> This is an interesting academic idea. But the way you implement it
> will immediately kill all existing full and SPV nodes (not really
> dead, rather like zombie as they can't send and receive any tx).

That's the whole point.  After a conventional hardfork everyone needs to 
upgrade, but there is no way to force users to upgrade.  A user who is 
simply unaware of the fork, or disagrees with the fork, uses the old 
client and the currency splits.

Under this proposal old clients effectively enter "zombie" mode, forcing 
users to upgrade.

--joe
2015-12-21  4:41   ` joe2015

  2015-12-30 23:49       ` Jonathan Toomim
  
From: Bob McElrath @ 2015-12-30 19:00 UTC (permalink / raw)
  To: joe2015;  bitcoin-dev

joe2015--- via bitcoin-dev [bitcoin-dev@lists•linuxfoundation.org] wrote:
> That's the whole point.  After a conventional hardfork everyone
> needs to upgrade, but there is no way to force users to upgrade.  A
> user who is simply unaware of the fork, or disagrees with the fork,
> uses the old client and the currency splits.
> 
> Under this proposal old clients effectively enter "zombie" mode,
> forcing users to upgrade.

This is a very complex way to enter zombie mode.

A simpler way is to track valid PoW chains by examining only the header, that
are rejected for other reasons.

Once a chain is seen to be 6 or more blocks ahead of my chain tip, we should
enter "zombie mode" and refuse to mine or relay, and alert the operator, because
we don't know what we're doing and we're out of date.  This way doesn't require
any modifications to block structure at all.

--
Cheers, Bob McElrath

"For every complex problem, there is a solution that is simple, neat, and wrong."
    -- H. L. Mencken 



 permalink raw reply	[|nested]
2015-12-30 19:00     ` Bob McElrath

  2015-12-30 23:56         ` Jonathan Toomim
  2015-12-31  0:04         ` Bob McElrath
  
From: Jonathan Toomim @ 2015-12-30 23:49 UTC (permalink / raw)
  To: Bitcoin Dev

[-- Attachment #1: Type: text/plain, Size: 3064 bytes --]

On Dec 30, 2015, at 11:00 AM, Bob McElrath via bitcoin-dev <bitcoin-dev@lists•linuxfoundation.org> wrote:

> joe2015--- via bitcoin-dev [bitcoin-dev@lists•linuxfoundation.org] wrote:
>> That's the whole point.  After a conventional hardfork everyone
>> needs to upgrade, but there is no way to force users to upgrade.  A
>> user who is simply unaware of the fork, or disagrees with the fork,
>> uses the old client and the currency splits.
>> 
>> Under this proposal old clients effectively enter "zombie" mode,
>> forcing users to upgrade.
> 
> This is a very complex way to enter zombie mode.


Another way you could make non-upgraded nodes enter zombie mode is to explicitly 51% attack the minority fork.

All soft forks are controlled, coordinated, developer-sanctioned 51% attacks against nodes that do not upgrade. The generalized softfork technique is a method of performing a soft fork that completely eliminates any usefulness to non-upgraded nodes while merge-mining another block structure to provide functionality to the nodes who have upgraded and know where to look for the new data.

Soft forks are "safe" forks because you can trust the miners to censor blocks and transactions that do not conform to the new consensus rules. Since we've been relying on the trustworthiness of miners during soft forks in the past (and it only failed us once!), why not

The generalized softfork method has the advantage of being merge-mined, so miners don't have to lose any revenue while performing this 51% attack against non-upgraded nodes. But then you're stuck with all of your transactions in a merge-mined/commitment-based data structure, which is a bit awkward and ugly. But you could avoid all of that code ugliness by just convincing the miners to donate some hashrate (say, 5.1% if the IsSupermajority threshold is 95%, or you could make it dynamic to save some money) to ensuring that the minority fork never has any transactions in the chain. That way, you can replace the everlasting code ugliness with a little bit of temporary sociopolitical ugliness. Fortunately, angry people are easier to ignore than ugly code. /s

Maybe we could call this a softly enforced hard fork? It's basically a combined hard fork for the supermajority and a soft fork to make the minority chain useless.

I don't personally think that these 51% attacks are useful or necessary. This is one of the main reasons why I don't like soft forks. I find them distasteful, and think that leaving minorities free to practice their own religions and blockchain rules is a good thing. But I could see how this could address some of the objections that others have raised about the dangers of hardforks, so I'm putting it out there.

> Once a chain is seen to be 6 or more blocks ahead of my chain tip, we should
> enter "zombie mode" and refuse to mine or relay

I like this method. However, it does have the problem of being voluntary. If nodes don't upgrade to a version that has the latent zombie gene long before a fork, then it does nothing.



[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 496 bytes --]

 permalink raw reply	[|nested]
2015-12-30 23:49       ` Jonathan Toomim

  2015-12-31  0:04         ` Bob McElrath
  
From: Jonathan Toomim @ 2015-12-30 23:56 UTC (permalink / raw)
  To: Bitcoin Dev


[-- Attachment #1.1: Type: text/plain, Size: 255 bytes --]


On Dec 30, 2015, at 3:49 PM, Jonathan Toomim <j@toom•im> wrote:

> Since we've been relying on the trustworthiness of miners during soft forks in the past (and it only failed us once!), why not

make it explicit?

(Sorry for the premature send.)

[-- Attachment #1.2: Type: text/html, Size: 1312 bytes --]

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 496 bytes --]

 permalink raw reply	[|nested]
2015-12-30 23:49       ` Jonathan Toomim
  2015-12-30 23:56         ` Jonathan Toomim

  2015-12-31  4:39           ` joe2015
  
From: Bob McElrath @ 2015-12-31  0:04 UTC (permalink / raw)
  To: Jonathan Toomim;  Bitcoin Dev

Jonathan Toomim [j@toom•im] wrote:
> 
> The generalized softfork method has the advantage of being merge-mined

That's an over-generalization.  There are two kinds of soft-forks WRT mining,
those which:

1. involve new validation rules by data-hiding from non-upgraded modes
    (e.g. extension blocks, generalized softfork)
2. involve NO new validation logic (e.g. P2SH)

Miners which are not validating transactions *should* be deprived of revenue,
because their role is transaction validation, not simply brute forcing sha256d.

So I'm very strongly against this "generalized softfork" idea -- I also don't
see how upgraded nodes and non-upgraded nodes can possibly end up with the same
UTXO set.

> > Once a chain is seen to be 6 or more blocks ahead of my chain tip, we should
> > enter "zombie mode" and refuse to mine or relay
> 
> I like this method. However, it does have the problem of being voluntary. If
> nodes don't upgrade to a version that has the latent zombie gene long before a
> fork, then it does nothing.

Which is why it should be put into core long before forks.  ;-)

--
Cheers, Bob McElrath

"For every complex problem, there is a solution that is simple, neat, and wrong."
    -- H. L. Mencken 



 permalink raw reply	[|nested]
2015-12-31  0:04         ` Bob McElrath

  2015-12-31 10:39             ` David Chan
  
From: joe2015 @ 2015-12-31  4:39 UTC (permalink / raw)
  To: Bob McElrath;  Bitcoin Dev

> So I'm very strongly against this "generalized softfork" idea -- I also 
> don't
> see how upgraded nodes and non-upgraded nodes can possibly end up with 
> the same
> UTXO set.

The only way for non-upgraded nodes to get the correct UTXO set is to 
upgrade.

It is important to keep in mind this was proposed as an alternative to a 
hardfork.  With a hardfork the UTXOs also diverge as upgraded and 
non-upgraded clients follow different chains.

--joe.


 permalink raw reply	[|nested]
2015-12-31  4:39           ` joe2015

  2015-12-31 11:32               ` joe2015
  
From: David Chan @ 2015-12-31 10:39 UTC (permalink / raw)
  To: joe2015;  Bitcoin Dev

The UTXO sets may diverge but they actually will be strict subsets/supersets of each other as no transaction would be invalid on one fork vs another unless the hard fork lasts longer than 100 blocks. 
This is of course specific to a block limit change hard fork. 



On 2015/12/31, at 13:39, joe2015--- via bitcoin-dev <bitcoin-dev@lists•linuxfoundation.org> wrote:

>> So I'm very strongly against this "generalized softfork" idea -- I also don't
>> see how upgraded nodes and non-upgraded nodes can possibly end up with the same
>> UTXO set.
> 
> The only way for non-upgraded nodes to get the correct UTXO set is to upgrade.
> 
> It is important to keep in mind this was proposed as an alternative to a hardfork.  With a hardfork the UTXOs also diverge as upgraded and non-upgraded clients follow different chains.
> 
> --joe.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


 permalink raw reply	[|nested]
2015-12-31 10:39             ` David Chan

  
From: joe2015 @ 2015-12-31 11:32 UTC (permalink / raw)
  To: David Chan;  Bitcoin Dev

On 2015-12-31 18:39, David Chan wrote:
> The UTXO sets may diverge but they actually will be strict
> subsets/supersets of each other as no transaction would be invalid on
> one fork vs another unless the hard fork lasts longer than 100 blocks.

The UTXO sets can also diverge thanks to double spends, i.e. A->B is 
confirmed on the old chain and A->C is confirmed on the new.

--joe.


 permalink raw reply	[|nested]
2015-12-20 10:56 [bitcoin-dev] Increasing the blocksize as a (generalized) softfork joe2015
                   ` (3 preceding siblings ...)
  2015-12-21  4:23 ` jl2012

  
From: Luke Dashjr @ 2016-01-04 21:53 UTC (permalink / raw)
  To: bitcoin-dev, joe2015

On Sunday, December 20, 2015 10:56:33 AM joe2015--- via bitcoin-dev wrote:
> "generalized" softfork.

FWIW, this is something I've been planning to proposed (in a nicer form) for a 
while, tentatively called a "soft hardfork" (or less-seriously a "softserve 
hardfork"). The big piece missing that I've been holding off on publishing it 
as a BIP until complete, is a planned-out defensive reaction for a community 
which wishes to reject the hardfork. I guess I should probably prioritise this 
a bit more now...

Luke


 permalink raw reply	[|nested]