BIP proposal: NODE_NETWORK_LIMITED service bits

BIP proposal: NODE_NETWORK_LIMITED service bits
2017-05-11 18:17 ` Gregory Maxwell
                   ` (2 more replies)
  
From: Jonas Schnelli @ 2017-05-11 15:13 UTC (permalink / raw)
  To: bitcoin-dev


[-- Attachment #1.1: Type: text/plain, Size: 318 bytes --]

Hi

Currently, pruned peers have no way how to signal their (valuable) service.
A BIP proposal to improve this (draft):
https://github.com/jonasschnelli/bips/wiki/NODE_NETWORK_LIMITED-BIP-DRAFT <https://github.com/jonasschnelli/bips/wiki/NODE_NETWORK_LIMITED-BIP-DRAFT>

Feedback is highly welcome.

</jonas>

[-- Attachment #1.2: Type: text/html, Size: 739 bytes --]

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

 permalink raw reply	[|nested]
2017-05-11 18:17 ` 
                   ` 
  
From: Jonas Schnelli @ 2017-05-11 15:13 UTC (permalink / raw)
  To: bitcoin-dev


[-- Attachment #1.1: Type: text/plain, Size: 318 bytes --]

Hi

Currently, pruned peers have no way how to signal their (valuable) service.
A BIP proposal to improve this (draft):
https://github.com/jonasschnelli/bips/wiki/NODE_NETWORK_LIMITED-BIP-DRAFT <https://github.com/jonasschnelli/bips/wiki/NODE_NETWORK_LIMITED-BIP-DRAFT>

Feedback is highly welcome.

</jonas>

[-- Attachment #1.2: Type: text/html, Size: 739 bytes --]

[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

 permalink raw reply	[|nested]
2017-05-11 15:13 [bitcoin-dev] BIP proposal: NODE_NETWORK_LIMITED service bits Jonas Schnelli
  2017-05-11 18:17 ` Gregory Maxwell

  2017-05-11 20:10   ` Jonas Schnelli
  2017-05-12  2:22 ` Gregory Maxwell
  
From: Luke Dashjr @ 2017-05-11 19:24 UTC (permalink / raw)
  To: bitcoin-dev, Jonas Schnelli

> A peer signaling NODE_NETWORK_LIMITED_LOW & NODE_NETWORK_LIMITED_HIGH MUST
> be capable of serving at least the last 7'056 blocks (~49 days)
> (NODE_NETWORK_LIMITED_HIGH's value ^2).

Is 49 days particularly useful? Would it be a problem to instead leave both-
bits undefined? I'm thinking this might be better as a way to indicate "7 
days, plus a deterministically chosen set of historical blocks"...

> Current Bitcoin-Core pruned full nodes guarantees a minimum of 288 blocks,
> thus allowing to signal NODE_NETWORK_LIMITED_LOW with the current minimum
> configuration.

This is technically true right now, but as soon as segwit activates, it will 
no longer be... Therefore, I suggest striking it from the BIP, expounding on 
it in greater detail, or making it true for the longer term.

> Peers following this BIP SHOULD connect a limited amount of their available
> outbound connections to peers signaling one or both of the
> NODE_NETWORK_LIMITED_* service bits if they expect to request less blocks
> than the signaled number.

This isn't entirely clear whether it refers to peers downloading blocks, or 
peers serving them. (I assume the former, but it should be clarified.)

> Light clients (and such) who are not checking the nServiceFlags (service
> bits) from a relayed addr-message may unwillingly connect to a pruned peer
> and ask for (filtered) blocks at a depth below their pruned depth.

Wouldn't this already be a problem, without the BIP?

Luke


 permalink raw reply	[|nested]
2017-05-11 19:24 ` Luke Dashjr

  2017-05-11 20:36     ` Aymeric Vitte
  
From: Jonas Schnelli @ 2017-05-11 20:10 UTC (permalink / raw)
  To: Luke Dashjr;  bitcoin-dev

[-- Attachment #1: Type: text/plain, Size: 1955 bytes --]


> Is 49 days particularly useful? Would it be a problem to instead leave both-
> bits undefined? I'm thinking this might be better as a way to indicate "7
> days, plus a deterministically chosen set of historical blocks"…

I though two service bits allow three states and we should define all three combinations.
But I guess an adequate „definition“ would be to reserve it for future „definitions“.
Or use Gregory's proposal of min 2016*2 blocks & keep it „undefined“ for now.

49 days was chosen to allow SPV peers to be „offline“ for a month and still be capable to catch-up with a peer pruned to a datadir of ~10GB.

> 
> This is technically true right now, but as soon as segwit activates, it will
> no longer be... Therefore, I suggest striking it from the BIP, expounding on
> it in greater detail, or making it true for the longer term.

AFAIK Core does also guaranteed the 288 blocks post segwit activation:
https://github.com/bitcoin/bitcoin/blob/08a7316c144f9f2516db8fa62400893f4358c5ae/src/validation.h#L204
But maybe I’m confused.

> 
>> Peers following this BIP SHOULD connect a limited amount of their available
>> outbound connections to peers signaling one or both of the
>> NODE_NETWORK_LIMITED_* service bits if they expect to request less blocks
>> than the signaled number.
> 
> This isn't entirely clear whether it refers to peers downloading blocks, or
> peers serving them. (I assume the former, but it should be clarified.)

Indeed. I’ll try to make that more clear.

> 
>> Light clients (and such) who are not checking the nServiceFlags (service
>> bits) from a relayed addr-message may unwillingly connect to a pruned peer
>> and ask for (filtered) blocks at a depth below their pruned depth.
> 
> Wouldn't this already be a problem, without the BIP?

AFAIK, Core does currently only relay NODE_NETWORK addresses.
But yes, It may be a problem already.

</jonas>


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

 permalink raw reply	[|nested]
2017-05-11 20:10   ` Jonas Schnelli

  2017-05-11 21:05       ` Eric Voskuil
  
From: Aymeric Vitte @ 2017-05-11 20:36 UTC (permalink / raw)
  To: Jonas Schnelli, Luke Dashjr;  bitcoin-dev

[-- Attachment #1: Type: text/plain, Size: 3181 bytes --]

Sorry again, is this discussion really serious?

In this thread, previous ones or others, the level of approximation is
obvious, often shadowed by useless maths/papers and strange calculations
that are not helping, at the end nobody knows what would happen "if",
how far the chain can roll back, etc

Then don't make pruning the default if it's the current concern, pruning
is of no use

Again, the priority should be to scale the full nodes


Le 11/05/2017 à 22:10, Jonas Schnelli via bitcoin-dev a écrit :
>> Is 49 days particularly useful? Would it be a problem to instead leave both-
>> bits undefined? I'm thinking this might be better as a way to indicate "7
>> days, plus a deterministically chosen set of historical blocks"…
> I though two service bits allow three states and we should define all three combinations.
> But I guess an adequate „definition“ would be to reserve it for future „definitions“.
> Or use Gregory's proposal of min 2016*2 blocks & keep it „undefined“ for now.
>
> 49 days was chosen to allow SPV peers to be „offline“ for a month and still be capable to catch-up with a peer pruned to a datadir of ~10GB.
>
>> This is technically true right now, but as soon as segwit activates, it will
>> no longer be... Therefore, I suggest striking it from the BIP, expounding on
>> it in greater detail, or making it true for the longer term.
> AFAIK Core does also guaranteed the 288 blocks post segwit activation:
> https://github.com/bitcoin/bitcoin/blob/08a7316c144f9f2516db8fa62400893f4358c5ae/src/validation.h#L204
> But maybe I’m confused.
>
>>> Peers following this BIP SHOULD connect a limited amount of their available
>>> outbound connections to peers signaling one or both of the
>>> NODE_NETWORK_LIMITED_* service bits if they expect to request less blocks
>>> than the signaled number.
>> This isn't entirely clear whether it refers to peers downloading blocks, or
>> peers serving them. (I assume the former, but it should be clarified.)
> Indeed. I’ll try to make that more clear.
>
>>> Light clients (and such) who are not checking the nServiceFlags (service
>>> bits) from a relayed addr-message may unwillingly connect to a pruned peer
>>> and ask for (filtered) blocks at a depth below their pruned depth.
>> Wouldn't this already be a problem, without the BIP?
> AFAIK, Core does currently only relay NODE_NETWORK addresses.
> But yes, It may be a problem already.
>
> </jonas>
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

-- 
Zcash wallets made simple: https://github.com/Ayms/zcash-wallets
Bitcoin wallets made simple: https://github.com/Ayms/bitcoin-wallets
Get the torrent dynamic blocklist: http://peersm.com/getblocklist
Check the 10 M passwords list: http://peersm.com/findmyass
Anti-spies and private torrents, dynamic blocklist: http://torrent-live.org
Peersm : http://www.peersm.com
torrent-live: https://github.com/Ayms/torrent-live
node-Tor : https://www.github.com/Ayms/node-Tor
GitHub : https://www.github.com/Ayms


[-- Attachment #2: Type: text/html, Size: 5338 bytes --]

 permalink raw reply	[|nested]
2017-05-11 20:36     ` Aymeric Vitte

  
From: Eric Voskuil @ 2017-05-11 21:05 UTC (permalink / raw)
  To: Aymeric Vitte, Jonas Schnelli, Luke Dashjr;  bitcoin-dev

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 05/11/2017 01:36 PM, Aymeric Vitte via bitcoin-dev wrote:
> Sorry again, is this discussion really serious?
> 
> In this thread, previous ones or others, the level of approximation
> is obvious, often shadowed by useless maths/papers and strange
> calculations that are not helping, at the end nobody knows what
> would happen "if", how far the chain can roll back, etc
> 
> Then don't make pruning the default if it's the current concern,
> pruning is of no use
> 
> Again, the priority should be to scale the full nodes

I agree. Every full node operator should (and likely would at some
point) simply never connect to, or relay the address of, any "peer"
advertising itself as diminished. Why on earth would a full node
operator want to encourage shrinking support for bootstrapping and
deep reorgs, resulting in greater load for himself. That's a race to
the bottom.

We are literally talking about $7.50 for the *entire chain* with
breathing room. If someone wants to save a few dollars they are better
off attempting to hide their pruning.

e

> Le 11/05/2017 à 22:10, Jonas Schnelli via bitcoin-dev a écrit :
>>> Is 49 days particularly useful? Would it be a problem to
>>> instead leave both- bits undefined? I'm thinking this might be
>>> better as a way to indicate "7 days, plus a deterministically
>>> chosen set of historical blocks"…
>> I though two service bits allow three states and we should define
>> all three combinations. But I guess an adequate „definition“
>> would be to reserve it for future „definitions“. Or use Gregory's
>> proposal of min 2016*2 blocks & keep it „undefined“ for now.
>> 
>> 49 days was chosen to allow SPV peers to be „offline“ for a month
>> and still be capable to catch-up with a peer pruned to a datadir
>> of ~10GB.
>> 
>>> This is technically true right now, but as soon as segwit
>>> activates, it will no longer be... Therefore, I suggest
>>> striking it from the BIP, expounding on it in greater detail,
>>> or making it true for the longer term.
>> AFAIK Core does also guaranteed the 288 blocks post segwit
>> activation: 
>> https://github.com/bitcoin/bitcoin/blob/08a7316c144f9f2516db8fa624008
93f4358c5ae/src/validation.h#L204
>>
>> 
But maybe I’m confused.
>> 
>>>> Peers following this BIP SHOULD connect a limited amount of
>>>> their available outbound connections to peers signaling one
>>>> or both of the NODE_NETWORK_LIMITED_* service bits if they
>>>> expect to request less blocks than the signaled number.
>>> This isn't entirely clear whether it refers to peers
>>> downloading blocks, or peers serving them. (I assume the
>>> former, but it should be clarified.)
>> Indeed. I’ll try to make that more clear.
>> 
>>>> Light clients (and such) who are not checking the
>>>> nServiceFlags (service bits) from a relayed addr-message may
>>>> unwillingly connect to a pruned peer and ask for (filtered)
>>>> blocks at a depth below their pruned depth.
>>> Wouldn't this already be a problem, without the BIP?
>> AFAIK, Core does currently only relay NODE_NETWORK addresses. But
>> yes, It may be a problem already.
>> 
>> </jonas>
>> 
>> 
>> 
>> _______________________________________________ bitcoin-dev
>> mailing list bitcoin-dev@lists•linuxfoundation.org 
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> 
> -- Zcash wallets made simple:
> https://github.com/Ayms/zcash-wallets Bitcoin wallets made simple:
> https://github.com/Ayms/bitcoin-wallets Get the torrent dynamic
> blocklist: http://peersm.com/getblocklist Check the 10 M passwords
> list: http://peersm.com/findmyass Anti-spies and private torrents,
> dynamic blocklist: http://torrent-live.org Peersm :
> http://www.peersm.com torrent-live:
> https://github.com/Ayms/torrent-live node-Tor :
> https://www.github.com/Ayms/node-Tor GitHub :
> https://www.github.com/Ayms
> 
> 
> 
> _______________________________________________ bitcoin-dev mailing
> list bitcoin-dev@lists•linuxfoundation.org 
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQEcBAEBCAAGBQJZFNHtAAoJEDzYwH8LXOFOYRwH/0By+TNSgnV6m4c7g1ZrjboG
8fZSeGaz7FXmAUZ69XMdQ1H+wlP0e4OAz9eRCcVqcn3K9OZJn++hbzI2K+zijyxZ
cpQjg/2dcTc4B0Z3PZdnuZx5mnHzavr/1vPlgOYla7AbYqcKB5dkq/HqgD6tFsJP
HXPClbEkVRF6UFP/7sDfzW8FMJycMSVcbEpuWAhcx2d+NusywWhbkuc5NiT9J1Ug
/3OFhHVJtd+rDl2B4iRHXHOhysUGffvmmLANZpPMcOgplM6Xwv7nMT34FV4HCdgs
Gyxc9GSFsD6xsOshBRPICtEZe+IDDb0cnOLjDdAnUnKeolUljFY52djSa300Fp0=
=REyc
-----END PGP SIGNATURE-----


 permalink raw reply	[|nested]
2017-05-11 15:13 [bitcoin-dev] BIP proposal: NODE_NETWORK_LIMITED service bits Jonas Schnelli
  2017-05-11 18:17 ` Gregory Maxwell
  2017-05-11 19:24 ` Luke Dashjr

  
From: Gregory Maxwell @ 2017-05-12  2:22 UTC ( / )
  To: Jonas Schnelli;  bitcoin-dev

On Thu, May 11, 2017 at 3:13 PM, Jonas Schnelli via bitcoin-dev
<bitcoin-dev@lists•linuxfoundation.org> wrote:
> Hi
>
> Currently, pruned peers have no way how to signal their (valuable) service.
> A BIP proposal to improve this (draft):
> https://github.com/jonasschnelli/bips/wiki/NODE_NETWORK_LIMITED-BIP-DRAFT

The instructions for relay addresses should not instruct you to relay
these addresses but rather that you should relay addresses you would
connect to, under the generalized assumption that if it is useful to
you it will be useful to others.

This avoids instructing someone who might not consider
non-node-network peers useful from being directed by the BIP to relay
things that they don't find useful. (In particular, it means that the
obvious implementation of just throwing out the 'useless' information
works fine.)  I think would better reflect what people are likely to
actually do.