Analysis of full-RBF deployment methods

Analysis of full-RBF deployment methods
2022-10-21  1:50 ` Antoine Riard
  
From: Dario Sneidermanis @ 2022-10-20 16:51 UTC (permalink / raw)
  To: Bitcoin Protocol Discussion

[-- Attachment #1: Type: text/plain, Size: 6050 bytes --]

Hello list,

Given that the release of 24.0 is upon us and there is little time to make a
complex decision regarding the deployment method of full-RBF, we've
documented
the different alternatives and their trade-offs. I hope this helps get to
the
best possible deployment!

Gist: https://gist.github.com/esneider/4eb16fcd959cb8c6b657c314442801ee

# Current deployment options

1. Antoine's PR #26305: leave 24.0 as is, and merge opt-out in 25.0 or
later.
2. Marco's PR #26287: revert opt-in full-RBF in 24.0, and give more time to
   figure out what's next.
3. Marco's PR #26287 + Antoine's PR #26305: revert opt-in full-RBF in 24.0,
and
   merge opt-out in 25.0 or later.
4. Marco's PR #26287 + Anthony's PR #26323 (just the date commitment):
revert
   opt-in full-RBF in 24.0, and commit in 25.0 or later to a later date for
   opt-out activation.
5. Anthony's PR #26323: revert opt-in full-RBF in 24.0, and commit in 24.0
to a
   later date for opt-out activation.

Notice that once full-RBF is fully deployed, having a config option to
disable
it is mostly a foot gun: you will only hurt yourself by missing some
transactions. Maybe options 4 and 5 could remove the flag altogether
instead of
making it opt-out.

There are a few more options, but I don't think they would reasonably have
any
consensus, so I trimmed them down to make it easier to process.


# Dimensions of analysis

1. Zero-conf apps immediately affected

    If we leave the flag for full-rbf in 24.0, zero-conf apps could be
    immediately affected. More specifically, as Anthony explained much more
    clearly [0], they would be in danger as soon as a relatively big mining
    pool operator enables the full-RBF flag.

    It turns out that the class of apps that could be immediately affected
(ie.
    apps that were directly or indirectly relying on the first-seen policy
in an
    adversarial setting) is larger than zero-conf apps, as exposed by Sergej
    [1]. Namely, the apps committing to an exchange rate before on-chain
funds
    are sent/finalized would start offering a free(ish) american call
option.

2. Predictable deployment date

    Committing to an activation date for full-rbf on the social layer (eg.
    "we'll merge the opt-out flag in 25.0") has the benefit of being
flexible in
    the event of new data points but becomes less predictable (both for
    applications and for full-rbf proponents).

    Committing to an activation date for full-rbf on the code has the
benefit
    that once node operators start deploying the code, the date is set in
stone,
    and we can reason about when full-RBF will be fully deployed and usable.

3. Code complexity

    Handling the commitment to a date in the code introduces further code
    complexity. In particular, it's a deployment mechanism that, as far as I
    know, hasn't been tried before, so we should be careful.

4. Smooth deployment

    Full-RBF deployment has two distinct phases when analyzing the adoption
in
    the transaction relaying layer. First, there will be multiple disjoint
    connected components of full-RBF nodes. Eventually, we'll get to a
    single(ish) connected component of full-RBF nodes.

    The first deployment phase is a bit chaotic and difficult to reason
about:
    nobody can rely on full-RBF actually working; if it coincides with a
    high-fees scenario, we'll get a big mempool divergence event, causing
many
    other issues and unreliability in the relaying and application layers.

    I'm calling smooth deployment to a deployment that minimizes the first
    phase, eg. by activating full-RBF simultaneously in as many
    transaction-relaying nodes as possible.

5. Time to figure out the right deployment

    Figuring out the right deployment method and timeline to activate
full-rbf
    might be more time-consuming than what we are willing to wait for the
stable
    release of 24.0. Decoupling the protection to zero-conf apps from
choosing a
    deployment method and an activation date for opt-out might be a good
idea.

I'm probably forgetting some dimensions here, but it may be enough to grasp
the
trade-offs between the different approaches.


# Comparison

Gist:
https://gist.github.com/esneider/4eb16fcd959cb8c6b657c314442801ee#comparison

# Timeline for full-RBF activation

If we make some UX trade-offs, Muun can be production ready with the
required
changes in 6 months. Having more time to avoid those trade-offs would be
preferable, but we can manage.

The larger application ecosystem may need a bit more time since they might
not
have the advantage of having been working on the required changes for a
while
already. Ideally, there should be enough time to reach out to affected
applications and let them make time to understand the impact, design
solutions,
implement them, and deploy them.

Finally, if a smooth deployment (as previously defined) is desired, we can
lock
an activation date in the code and give relaying nodes enough time to
upgrade
before activation. Assuming that the adoption of future releases remains
similar
to previous ones [2], one release cycle should get us to 22% adoption, two
release cycles to 61% adoption, and three release cycles to 79% adoption.
Assuming a uniform adoption distribution, the probability of an 8-connection
relaying node not being connected to any full-RBF node after one release
cycle
will be 0.14. After two cycles, it will be 0.00054, and after three cycles,
it
will be 0.0000038. Looking at these numbers, it would seem that a single
release
cycle will be too little time, but two release cycles may be enough.

Cheers,
Dario


[0]
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-October/021031.html
[1]
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-October/021056.html
[2] https://luke.dashjr.org/programs/bitcoin/files/charts/software.html
[Marco's PR #26287] https://github.com/bitcoin/bitcoin/pull/26287
[Antoine's PR #26305] https://github.com/bitcoin/bitcoin/pull/26305
[Anthony's PR #26323] https://github.com/bitcoin/bitcoin/pull/26323

[-- Attachment #2: Type: text/html, Size: 7245 bytes --]

 permalink raw reply	[|nested]
2022-10-20 16:51 [bitcoin-dev] Analysis of full-RBF deployment methods Dario Sneidermanis

  2022-10-21 21:13   ` Dario Sneidermanis
  
From: Antoine Riard @ 2022-10-21  1:50 UTC ( / )
  To: Dario Sneidermanis, Bitcoin Protocol Discussion



Hi Dario,

Thanks for this analysis of full-RBF deployment methods!

The subject was widely discussed at today Bitcoin Core IRC meetings:
https://gnusha.org/bitcoin-core-dev/2022-10-20.log

Personally, I still think deferring full-rbf deployment, while it sounds
reasonable to let existing services and applications adapt their software
and business models, doesn't come risk-free for the contracting protocols
and multi-party applications affected by the pinning DoS vector. Deferring
ad vitam aeternam left them exposed to disruptions when their traffic
volume would start to be significant. While those use-cases
(splicing/dual-channels/collaborative constructions) were mostly vaporware
a year ago when I raised the issue, it turns out they have become a far
more tangible reality today. Beyond the 3 coinjoins services
(Wasabi/Joinmarket/Whirlpool), we have new things like ln-vortex, or
Phoenix wallet and some LDK users planning to use dual-funded soon.

I'm fully aware we still have community stakeholders opposing full-rbf on
principle, I think this is highly valuable to exchange more perspectives on
the subject, hopefully to harmonize understandings, and work altogether on
creative solutions to accommodate as many use-cases as we can, in a
compatible way.

I don't know what the best full-rbf deployment method is, though one
offering clear visibility and predictability to node operators, with a flag
day activation like #26323 sounds like a reasonable one. As you said, I
believe 6 months from now is far too short, maybe something like 12 months
or 16 months (or more ?) is adequate. At least I fully rejoin on the risk
of merchants centralization if we favor a timeline only favoring entities
with sufficient manpower.

I'm still looking forward to having more forums and communication channels
between business/services operators and protocol developers, it sounds like
functional responsibilities between protocol and application layers could
be better clarified. However, I don't know if it should be the
responsibility of developers to solve every operational risk encumbered by
a Bitcoin business, like FX risk. I don't deny the interdependency between
network policy rules and business risk, I'm just saying Bitcoin protocol
developers have already heavily loaded engineering priorities between
solving the half of dozen of Lightning vulnerabilities, working on the next
consensus changes or reviewing modularity refactoring of Bitcoin Core to
extend the feature set in a soft way (among tons of other examples).

All speaking for myself, looking forward to having more perspective
expressed!

Best,
Antoine

Le jeu. 20 oct. 2022 à 12:56, Dario Sneidermanis via bitcoin-dev <
bitcoin-dev@lists•linuxfoundation.org> a écrit :

> Hello list,
>
> Given that the release of 24.0 is upon us and there is little time to make
> a
> complex decision regarding the deployment method of full-RBF, we've
> documented
> the different alternatives and their trade-offs. I hope this helps get to
> the
> best possible deployment!
>
> Gist: https://gist.github.com/esneider/4eb16fcd959cb8c6b657c314442801ee
>
> # Current deployment options
>
> 1. Antoine's PR #26305: leave 24.0 as is, and merge opt-out in 25.0 or
> later.
> 2. Marco's PR #26287: revert opt-in full-RBF in 24.0, and give more time to
>    figure out what's next.
> 3. Marco's PR #26287 + Antoine's PR #26305: revert opt-in full-RBF in
> 24.0, and
>    merge opt-out in 25.0 or later.
> 4. Marco's PR #26287 + Anthony's PR #26323 (just the date commitment):
> revert
>    opt-in full-RBF in 24.0, and commit in 25.0 or later to a later date for
>    opt-out activation.
> 5. Anthony's PR #26323: revert opt-in full-RBF in 24.0, and commit in 24.0
> to a
>    later date for opt-out activation.
>
> Notice that once full-RBF is fully deployed, having a config option to
> disable
> it is mostly a foot gun: you will only hurt yourself by missing some
> transactions. Maybe options 4 and 5 could remove the flag altogether
> instead of
> making it opt-out.
>
> There are a few more options, but I don't think they would reasonably have
> any
> consensus, so I trimmed them down to make it easier to process.
>
>
> # Dimensions of analysis
>
> 1. Zero-conf apps immediately affected
>
>     If we leave the flag for full-rbf in 24.0, zero-conf apps could be
>     immediately affected. More specifically, as Anthony explained much more
>     clearly [0], they would be in danger as soon as a relatively big mining
>     pool operator enables the full-RBF flag.
>
>     It turns out that the class of apps that could be immediately affected
> (ie.
>     apps that were directly or indirectly relying on the first-seen policy
> in an
>     adversarial setting) is larger than zero-conf apps, as exposed by
> Sergej
>     [1]. Namely, the apps committing to an exchange rate before on-chain
> funds
>     are sent/finalized would start offering a free(ish) american call
> option.
>
> 2. Predictable deployment date
>
>     Committing to an activation date for full-rbf on the social layer (eg.
>     "we'll merge the opt-out flag in 25.0") has the benefit of being
> flexible in
>     the event of new data points but becomes less predictable (both for
>     applications and for full-rbf proponents).
>
>     Committing to an activation date for full-rbf on the code has the
> benefit
>     that once node operators start deploying the code, the date is set in
> stone,
>     and we can reason about when full-RBF will be fully deployed and
> usable.
>
> 3. Code complexity
>
>     Handling the commitment to a date in the code introduces further code
>     complexity. In particular, it's a deployment mechanism that, as far as
> I
>     know, hasn't been tried before, so we should be careful.
>
> 4. Smooth deployment
>
>     Full-RBF deployment has two distinct phases when analyzing the
> adoption in
>     the transaction relaying layer. First, there will be multiple disjoint
>     connected components of full-RBF nodes. Eventually, we'll get to a
>     single(ish) connected component of full-RBF nodes.
>
>     The first deployment phase is a bit chaotic and difficult to reason
> about:
>     nobody can rely on full-RBF actually working; if it coincides with a
>     high-fees scenario, we'll get a big mempool divergence event, causing
> many
>     other issues and unreliability in the relaying and application layers.
>
>     I'm calling smooth deployment to a deployment that minimizes the first
>     phase, eg. by activating full-RBF simultaneously in as many
>     transaction-relaying nodes as possible.
>
> 5. Time to figure out the right deployment
>
>     Figuring out the right deployment method and timeline to activate
> full-rbf
>     might be more time-consuming than what we are willing to wait for the
> stable
>     release of 24.0. Decoupling the protection to zero-conf apps from
> choosing a
>     deployment method and an activation date for opt-out might be a good
> idea.
>
> I'm probably forgetting some dimensions here, but it may be enough to
> grasp the
> trade-offs between the different approaches.
>
>
> # Comparison
>
> Gist:
> https://gist.github.com/esneider/4eb16fcd959cb8c6b657c314442801ee#comparison
>
> # Timeline for full-RBF activation
>
> If we make some UX trade-offs, Muun can be production ready with the
> required
> changes in 6 months. Having more time to avoid those trade-offs would be
> preferable, but we can manage.
>
> The larger application ecosystem may need a bit more time since they might
> not
> have the advantage of having been working on the required changes for a
> while
> already. Ideally, there should be enough time to reach out to affected
> applications and let them make time to understand the impact, design
> solutions,
> implement them, and deploy them.
>
> Finally, if a smooth deployment (as previously defined) is desired, we can
> lock
> an activation date in the code and give relaying nodes enough time to
> upgrade
> before activation. Assuming that the adoption of future releases remains
> similar
> to previous ones [2], one release cycle should get us to 22% adoption, two
> release cycles to 61% adoption, and three release cycles to 79% adoption.
> Assuming a uniform adoption distribution, the probability of an
> 8-connection
> relaying node not being connected to any full-RBF node after one release
> cycle
> will be 0.14. After two cycles, it will be 0.00054, and after three
> cycles, it
> will be 0.0000038. Looking at these numbers, it would seem that a single
> release
> cycle will be too little time, but two release cycles may be enough.
>
> Cheers,
> Dario
>
>
> [0]
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-October/021031.html
> [1]
> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-October/021056.html
> [2] https://luke.dashjr.org/programs/bitcoin/files/charts/software.html
> [Marco's PR #26287] https://github.com/bitcoin/bitcoin/pull/26287
> [Antoine's PR #26305] https://github.com/bitcoin/bitcoin/pull/26305
> [Anthony's PR #26323] https://github.com/bitcoin/bitcoin/pull/26323
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
2022-10-21  1:50 ` Antoine Riard

  2022-10-23 23:10     ` Antoine Riard
  
From: Dario Sneidermanis @ 2022-10-21 21:13 UTC (permalink / raw)
  To: Antoine Riard;  Bitcoin Protocol Discussion

[-- Attachment #1: Type: text/plain, Size: 3593 bytes --]

Hello Antoine,

Thanks for taking the time to answer every email with detailed analysis! I
can
see it's a lot of work. I'll answer inline.

On Thu, Oct 20, 2022 at 10:50 PM Antoine Riard <antoine.riard@gmail•com>
wrote:
> Personally, I still think deferring full-rbf deployment, while it sounds
> reasonable to let existing services and applications adapt their software
and
> business models, doesn't come risk-free for the contracting protocols and
> multi-party applications affected by the pinning DoS vector. Deferring ad
> vitam aeternam left them exposed to disruptions when their traffic volume
> would start to be significant. While those use-cases
> (splicing/dual-channels/collaborative constructions) were mostly
vaporware a
> year ago when I raised the issue, it turns out they have become a far more
> tangible reality today. Beyond the 3 coinjoins services
> (Wasabi/Joinmarket/Whirlpool), we have new things like ln-vortex, or
Phoenix
> wallet and some LDK users planning to use dual-funded soon.

To solve the attack you described in [0], collaborative transaction
protocols
(such as dual-funded channels) need a *reliable* way to replace
transactions.
Otherwise, protocol parties using full-RBF may see replacements succeed in
their
own mempool, only to find out they weren't relayed to a miner once it's too
late
(ie. once the replacement that won is mined).

I'm calling a full-RBF deployment reliable to the point at which any
full-RBF-enabled node can broadcast a replacement and get it relayed all
the way
to a miner in a reliable manner (ie. with high-enough probability).

Even if we deployed opt-out (or mandatory!) full-RBF now and miners adopted
it
immediately, it would take almost a year (assuming normal deployment times)
for
it to be sufficiently deployed in the relaying layer to be considered
reliable.
An opt-in full-RBF deployment, as currently proposed (ie. without #25600),
has
very little chance of getting us nowhere near that kind of adoption.

Notice that #26323 (option 5 in the OP) has the advantage of getting us to a
reliable full-RBF network the fastest (in particular, much faster than the
current opt-in deployment) while not threatening zero-conf applications
until
the activation time. That is, #26323 gives us a way in which we don't need
to
choose between the security of one use case versus the other. We can have
both.

> I'm still looking forward to having more forums and communication channels
> between business/services operators and protocol developers, it sounds
like
> functional responsibilities between protocol and application layers could
be
> better clarified. However, I don't know if it should be the
responsibility of
> developers to solve every operational risk encumbered by a Bitcoin
business,
> like FX risk. I don't deny the interdependency between network policy
rules
> and business risk, I'm just saying Bitcoin protocol developers have
already
> heavily loaded engineering priorities between solving the half of dozen of
> Lightning vulnerabilities, working on the next consensus changes or
reviewing
> modularity refactoring of Bitcoin Core to extend the feature set in a
soft way
> (among tons of other examples).

I don't think asking for a predictable deployment timeline for a change that
would put some applications at increased risk could be described as
burdening
the developers with solving every operational risk. This deployment method
comparison's goal was precisely to soften the burden on core devs.

Cheers,
Dario

[0]
https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html

[-- Attachment #2: Type: text/html, Size: 4061 bytes --]

 permalink raw reply	[|nested]
2022-10-21 21:13   ` Dario Sneidermanis

  
From: Antoine Riard @ 2022-10-23 23:10 UTC (permalink / raw)
  To: Dario Sneidermanis;  Bitcoin Protocol Discussion

[-- Attachment #1: Type: text/plain, Size: 6782 bytes --]

Hi Dario,

Thanks for providing more thoughts to the discussion!

> Notice that #26323 (option 5 in the OP) has the advantage of getting us
to a
> reliable full-RBF network the fastest (in particular, much faster than the
> current opt-in deployment) while not threatening zero-conf applications
> until
> the activation time. That is, #26323 gives us a way in which we don't need
> to
> choose between the security of one use case versus the other. We can have
> both.

For sure, contracting protocols and multi-party applications exposed by the
lack of full-rbf are still young overall, though as they attract more
volume they're also likely to become honeypots for any competing services
providers interested to hijack economic traffic (kinda the same concern
than channel jamming...) At the same time, we still have 0confs services
more exposed by full-rbf, a bit stuck between Scylla and Charybdis.

As commented on #26323, I'm personally fine with this approach, and I fully
opine that providing a clear and predictable time point to 0confs operators
is very valuable. Even more, I think May 1st 2023, is a bit too early,
10-12 months sounds more reasonable.

At the same time, I believe it's the opinion of a few developers and other
Bitcoin service operators that the Core project is taking too much
responsibility in taking for the network by shipping full-rbf=true.

(Really I'm 50/50 between those 2 opinions, as I'm the author of both
#26305 and #25600 and concept ACK on #26323, and any process forward would
sounds good to me)

> I don't think asking for a predictable deployment timeline for a change
that
> would put some applications at increased risk could be described as
> burdening
> the developers with solving every operational risk. This deployment method
> comparison's goal was precisely to soften the burden on core devs.

I can understand the confusion here. As it has been discussed on your
original thread, from my comprehension, the idea has been raised of a
optech working group or something to build collaboration between wallet
devs, merchant devs and protocol devs around "Bitcoin payment" issues like
FX risk, additional layers of security for 0confs, RBF and CPFP, etc [0].
While again, I reassert that such a multi-stakeholder forum could be really
fruitful for the ecosystem at large, I don't know if it should be a
prerequisite that we solve all the potential payment issues before
proceeding with full-rbf deployment. However I'm keeping aware about the
interdependency between full-rbf and operational, legal and business issues
that one encounters running a Bitcoin merchant/service, not easy to make
everything works I can guess.

[0]
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-October/021076.html

Best,
Antoine

Le ven. 21 oct. 2022 à 17:13, Dario Sneidermanis <dario@muun•com> a écrit :

> Hello Antoine,
>
> Thanks for taking the time to answer every email with detailed analysis! I
> can
> see it's a lot of work. I'll answer inline.
>
> On Thu, Oct 20, 2022 at 10:50 PM Antoine Riard <antoine.riard@gmail•com>
> wrote:
> > Personally, I still think deferring full-rbf deployment, while it sounds
> > reasonable to let existing services and applications adapt their
> software and
> > business models, doesn't come risk-free for the contracting protocols and
> > multi-party applications affected by the pinning DoS vector. Deferring ad
> > vitam aeternam left them exposed to disruptions when their traffic volume
> > would start to be significant. While those use-cases
> > (splicing/dual-channels/collaborative constructions) were mostly
> vaporware a
> > year ago when I raised the issue, it turns out they have become a far
> more
> > tangible reality today. Beyond the 3 coinjoins services
> > (Wasabi/Joinmarket/Whirlpool), we have new things like ln-vortex, or
> Phoenix
> > wallet and some LDK users planning to use dual-funded soon.
>
> To solve the attack you described in [0], collaborative transaction
> protocols
> (such as dual-funded channels) need a *reliable* way to replace
> transactions.
> Otherwise, protocol parties using full-RBF may see replacements succeed in
> their
> own mempool, only to find out they weren't relayed to a miner once it's
> too late
> (ie. once the replacement that won is mined).
>
> I'm calling a full-RBF deployment reliable to the point at which any
> full-RBF-enabled node can broadcast a replacement and get it relayed all
> the way
> to a miner in a reliable manner (ie. with high-enough probability).
>
> Even if we deployed opt-out (or mandatory!) full-RBF now and miners
> adopted it
> immediately, it would take almost a year (assuming normal deployment
> times) for
> it to be sufficiently deployed in the relaying layer to be considered
> reliable.
> An opt-in full-RBF deployment, as currently proposed (ie. without #25600),
> has
> very little chance of getting us nowhere near that kind of adoption.
>
> Notice that #26323 (option 5 in the OP) has the advantage of getting us to
> a
> reliable full-RBF network the fastest (in particular, much faster than the
> current opt-in deployment) while not threatening zero-conf applications
> until
> the activation time. That is, #26323 gives us a way in which we don't need
> to
> choose between the security of one use case versus the other. We can have
> both.
>
> > I'm still looking forward to having more forums and communication
> channels
> > between business/services operators and protocol developers, it sounds
> like
> > functional responsibilities between protocol and application layers
> could be
> > better clarified. However, I don't know if it should be the
> responsibility of
> > developers to solve every operational risk encumbered by a Bitcoin
> business,
> > like FX risk. I don't deny the interdependency between network policy
> rules
> > and business risk, I'm just saying Bitcoin protocol developers have
> already
> > heavily loaded engineering priorities between solving the half of dozen
> of
> > Lightning vulnerabilities, working on the next consensus changes or
> reviewing
> > modularity refactoring of Bitcoin Core to extend the feature set in a
> soft way
> > (among tons of other examples).
>
> I don't think asking for a predictable deployment timeline for a change
> that
> would put some applications at increased risk could be described as
> burdening
> the developers with solving every operational risk. This deployment method
> comparison's goal was precisely to soften the burden on core devs.
>
> Cheers,
> Dario
>
> [0]
> https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
>

[-- Attachment #2: Type: text/html, Size: 7509 bytes --]

 permalink raw reply	[|nested]