Would anyone object to adding a dlopen message hook system?

Would anyone object to adding a dlopen message hook system?
2017-08-13 20:00 ` Jonas Schnelli
  
From: Erik Aronesty @ 2017-08-13 18:46 UTC (permalink / raw)
  To: Bitcoin Protocol Discussion

[-- Attachment #1: Type: text/plain, Size: 2310 bytes --]

I was thinking about something like this that could add the ability for
module extensions in the core client.

When messages are received, modules hooks are called with the message data.


They can then handle, mark the peer invalid, push a message to the peer or
pass through an alternate command.  Also, modules could have their own
private commands prefixed by "x:" or something like that.

The idea is that the base P2P layer is left undisturbed, but there is now a
way to create "enhanced features" that some peers support.

My end goal is to support using lightning network micropayments to allow
people to pay for better node access - creating a market for node services.


But I don't think this should be "baked in" to core.   Nor do I think it
should be a "patch".   It should be a linked-in module, optionally compiled
and added to bitcoin conf, then loaded via dlopen().    Modules should be
slightly robust to Bitcoin versions changing out from under them, but not
if the network layer is changed.   This can be ensured by a) keeping a
module version number, and b) treating module responses as if they were
just received from the network.   Any module incompatibility should throw
an exception...ensuring broken peers don't stay online.

In general I think the core reference would benefit from the ability to
create subnetworks within the Bitcoin ecosystem.   Right now, we have two
choices... full node and get slammed with traffic, or listen-only node, and
do nothing.

Adding a module/hook system would allow a complex ecosystem of
participation - and it would seem to be far more robust in the long term.

Something like this???

class MessageHookIn {
public:
    int hookversion;
    int64_t nodeid;
    int nVersion;
    int64_t serviceflags;
    const char *strCommand;
    const char *nodeaddr;
    const char *vRecv;
    int vRecvLen;
    int64_t nTimeReceived;
};

class MessageHookOut {
public:
    int hookversion;
    int misbehaving;
    const char *logMsg;
    const char *pushCommand;
    const unsigned char *pushData;
    int pushDataLen;
    const char *passCommand;
    CDataStream passStream;
};

class MessageHook {
public:
    int hookversion;
    std::string name;
    typedef bool (*HandlerType)(const MessageHookIn *in, MessageHookOut
*out);
    HandlerType handle;
};

[-- Attachment #2: Type: text/html, Size: 2900 bytes --]

 permalink raw reply	[|nested]
2017-08-13 18:46 [bitcoin-dev] Would anyone object to adding a dlopen message hook system? Erik Aronesty

  2017-08-13 20:56   ` Mark Friedenbach
  
From: Jonas Schnelli @ 2017-08-13 20:00 UTC (permalink / raw)
  To: Erik Aronesty, Bitcoin Protocol Discussion

[-- Attachment #1: Type: text/plain, Size: 1923 bytes --]

Hi Erik

Thanks for your proposal.
In general, modularisation is a good thing, though proposing core to add modules wie dlopen() seems the wrong direction.
Core already has the problem of running to many things in the same process. The consensus logic, p2p system as well as the wallet AND the GUI do all share the same process (!).

A module approach like you describe would be a security nightmare (and Core is currently in the process of separating out the wallet and the GUI into its own process).

What does speak against using the existing IPC interfaces like RPC/ZMQ?
RPC can be bidirectional using long poll.

/jonas

> I was thinking about something like this that could add the ability for module extensions in the core client.
> 
> When messages are received, modules hooks are called with the message data.
> 
> They can then handle, mark the peer invalid, push a message to the peer or pass through an alternate command.  Also, modules could have their own private commands prefixed by "x:" or something like that.
> 
> The idea is that the base P2P layer is left undisturbed, but there is now a way to create "enhanced features" that some peers support.
> 
> My end goal is to support using lightning network micropayments to allow people to pay for better node access - creating a market for node services.
> 
> But I don't think this should be "baked in" to core.   Nor do I think it should be a "patch".   It should be a linked-in module, optionally compiled and added to bitcoin conf, then loaded via dlopen().    Modules should be slightly robust to Bitcoin versions changing out from under them, but not if the network layer is changed.   This can be ensured by a) keeping a module version number, and b) treating module responses as if they were just received from the network.   Any module incompatibility should throw an exception...ensuring broken peers don't stay online.


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

 permalink raw reply	[|nested]
2017-08-13 20:00 ` Jonas Schnelli

  2017-08-15  1:33     ` Erik Aronesty
  
From: Mark Friedenbach @ 2017-08-13 20:56 UTC (permalink / raw)
  To: Jonas Schnelli, Bitcoin Protocol Discussion

Jonas, I think his proposal is to enable extending the P2P layer, e.g.
adding new message types. Are you suggesting having externalized
message processing? That could be done via RPC/ZMQ while opening up a
much more narrow attack surface than dlopen, although I imagine such
an interface would require a very complex API specification.

On Sun, Aug 13, 2017 at 1:00 PM, Jonas Schnelli via bitcoin-dev
<bitcoin-dev@lists•linuxfoundation.org> wrote:
> Hi Erik
>
> Thanks for your proposal.
> In general, modularisation is a good thing, though proposing core to add modules wie dlopen() seems the wrong direction.
> Core already has the problem of running to many things in the same process. The consensus logic, p2p system as well as the wallet AND the GUI do all share the same process (!).
>
> A module approach like you describe would be a security nightmare (and Core is currently in the process of separating out the wallet and the GUI into its own process).
>
> What does speak against using the existing IPC interfaces like RPC/ZMQ?
> RPC can be bidirectional using long poll.
>
> /jonas
>
>> I was thinking about something like this that could add the ability for module extensions in the core client.
>>
>> When messages are received, modules hooks are called with the message data.
>>
>> They can then handle, mark the peer invalid, push a message to the peer or pass through an alternate command.  Also, modules could have their own private commands prefixed by "x:" or something like that.
>>
>> The idea is that the base P2P layer is left undisturbed, but there is now a way to create "enhanced features" that some peers support.
>>
>> My end goal is to support using lightning network micropayments to allow people to pay for better node access - creating a market for node services.
>>
>> But I don't think this should be "baked in" to core.   Nor do I think it should be a "patch".   It should be a linked-in module, optionally compiled and added to bitcoin conf, then loaded via dlopen().    Modules should be slightly robust to Bitcoin versions changing out from under them, but not if the network layer is changed.   This can be ensured by a) keeping a module version number, and b) treating module responses as if they were just received from the network.   Any module incompatibility should throw an exception...ensuring broken peers don't stay online.
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>


 permalink raw reply	[|nested]
2017-08-13 20:56   ` Mark Friedenbach

       [not found]       ` <CANAdVnpvEUBWbPa5BUOift-2R783Kc7fKa7xdLkqSgpqCaTp1g@mail.gmail.com>
  
From: Erik Aronesty @ 2017-08-15  1:33 UTC (permalink / raw)
  To: Mark Friedenbach;  Bitcoin Protocol Discussion

[-- Attachment #1: Type: text/plain, Size: 3020 bytes --]

Actually the more I think about it, the more I realize that all I need is
to listen on a new port, and use the RPC api to affect Bitcoin:

- ban a peer (# of hours)
- unban a peer (# of hours)

As long as I have those two functions, I can do everything I need.

On Sun, Aug 13, 2017 at 4:56 PM, Mark Friedenbach <mark@friedenbach•org>
wrote:

> Jonas, I think his proposal is to enable extending the P2P layer, e.g.
> adding new message types. Are you suggesting having externalized
> message processing? That could be done via RPC/ZMQ while opening up a
> much more narrow attack surface than dlopen, although I imagine such
> an interface would require a very complex API specification.
>
> On Sun, Aug 13, 2017 at 1:00 PM, Jonas Schnelli via bitcoin-dev
> <bitcoin-dev@lists•linuxfoundation.org> wrote:
> > Hi Erik
> >
> > Thanks for your proposal.
> > In general, modularisation is a good thing, though proposing core to add
> modules wie dlopen() seems the wrong direction.
> > Core already has the problem of running to many things in the same
> process. The consensus logic, p2p system as well as the wallet AND the GUI
> do all share the same process (!).
> >
> > A module approach like you describe would be a security nightmare (and
> Core is currently in the process of separating out the wallet and the GUI
> into its own process).
> >
> > What does speak against using the existing IPC interfaces like RPC/ZMQ?
> > RPC can be bidirectional using long poll.
> >
> > /jonas
> >
> >> I was thinking about something like this that could add the ability for
> module extensions in the core client.
> >>
> >> When messages are received, modules hooks are called with the message
> data.
> >>
> >> They can then handle, mark the peer invalid, push a message to the peer
> or pass through an alternate command.  Also, modules could have their own
> private commands prefixed by "x:" or something like that.
> >>
> >> The idea is that the base P2P layer is left undisturbed, but there is
> now a way to create "enhanced features" that some peers support.
> >>
> >> My end goal is to support using lightning network micropayments to
> allow people to pay for better node access - creating a market for node
> services.
> >>
> >> But I don't think this should be "baked in" to core.   Nor do I think
> it should be a "patch".   It should be a linked-in module, optionally
> compiled and added to bitcoin conf, then loaded via dlopen().    Modules
> should be slightly robust to Bitcoin versions changing out from under them,
> but not if the network layer is changed.   This can be ensured by a)
> keeping a module version number, and b) treating module responses as if
> they were just received from the network.   Any module incompatibility
> should throw an exception...ensuring broken peers don't stay online.
> >
> >
> > _______________________________________________
> > bitcoin-dev mailing list
> > bitcoin-dev@lists•linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> >
>

[-- Attachment #2: Type: text/html, Size: 3936 bytes --]

 permalink raw reply	[|nested]
[not found]       ` <CANAdVnpvEUBWbPa5BUOift-2R783Kc7fKa7xdLkqSgpqCaTp1g@mail.gmail.com>

  
From: Erik Aronesty @ 2017-08-15  4:44 UTC ( / )
  To: Julian Haight;  Bitcoin Protocol Discussion



The idea is that some peers, when you connect to them will work fine for
some time, but you need to find out the rate for services and send a
micropayment to maintain the connection.   This creates an optional pay
layer for high quality services, and also creates DDOS resistance in this
fallback layer.