Building c-lightning plugin with .NET
Tags: lightningnetwork, programming, csharp, dotnet
Published: 2022-03-27 10:53:58

> Update 2022/05/20: Based on the methodology described in this article, I have created a package for building your own plugin. please check out [DotNetLightning.ClnRpc](https://github.com/joemphilips/DotNetLightning/)


Recently, I have been modifying [my own Lapps](https://github.com/bitbankinc/NLoop) to work as a [c-lightning plugin](https://github.com/ElementsProject/lightning/blob/master/doc/PLUGINS.md) in order to make them work with not only with [LND](https://github.com/lightningnetwork/lnd) but also with [c-lightning]((https://github.com/ElementsProject/lightning) ) ([What are Lapps?](https://blockonomi.com/what-are-lapps/)).


I couldn't find someone doing it with .NET/C#/F#, so I will leave here a guideline for how to.

I also made [a sample application](https://github.com/joemphilips/HelloWorldPlugin), so if you are more interested in the sample than the explanation, please go there. My application is in F#, but this sample is written in C#. The language is different, but the basic approach is the same in both cases.

In this case, we will assume that there is a single class named `MyAwesomeRpcServer`, that has method handlers for RPC invocation. It starts out like this    

```csharp

using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
using StreamJsonRpc;

namespace HelloWorldPlugin.Server
{
  public class MyAwesomeRpcServerOptions
  {
    public string GreeterName { get; set; } = "World";
    public bool IsInitiated { get; set; }
  }

  public class MyAwesomeRpcServer
  {
    private readonly MyAwesomeRpcServerOptions _opts;

    public MyAwesomeRpcServer(MyAwesomeRpcServerOptions opts)
    {
      _opts = opts;
    }

    [JsonRpcMethod("hello")]
    public async Task<string> HelloAsync(string name)
    {
      return $"hello!! {name}! This is {_opts.GreeterName} !!";
    }
  }
}
```

The requirements are

1. We want to expose an rpc method called hello.
2. We want to return a message containing `GreeterName` passed as a command line argument
3. We want to return a message that includes the name passed as an RPC parameter from the User.
4. The number of exposed RPC may increase in the future.

### 1. Create a JsonRPCServer using StreamJsonRPC.

First, create an application that acts as an ordinary JsonRPC 2.0 server.
The standard way to do this is to use the [StreamJsonRpc](https://github.com/microsoft/vs-streamjsonrpc) library authored by MS.

If you want to use it as a normal web server over HTTP, [this sample](https://github.com/tpeczek/Demo.AspNetCore.StreamJsonRpc) is a good reference.
However, the c-lightning plugin communicates with `lightningd` itself via standard input/output, so this time we will use `Console.In` and `Console.Out` for transport.

It looks like this

```csharp
var rpcServer = new MyAwesomeRpcServer();
var formatter = new JsonMessageFormatter();
var handler = new NewLineDelimitedMessageHandler(Console.OpenStandardOutput(), Console.OpenStandardInput(), formatter);
var rpc = new JsonRpc(handler);
rpc.AddLocalRpcTarget(rpcServer, new JsonRpcTargetOptions());
rpc.ExceptionStrategy = ExceptionProcessing.CommonErrorData;
rpc.StartListening();
```

The Json RPC 2.0 specification does not specify the way of separating each messages as well as transports, so we must choose a suitable one.
Since the c-lightning plugin uses newlines to delimit messages, we use `NewLineDelimitedMessageHandler` as a message handler.
Also, if the communication partner is not C# (which is not in our case), it is a good to specify `ExceptionStrategy` as `ExceptionProcessing.CommonErrorData`.

### 2. Make read/write operations thread-safe.

Normally, the RPC server runs in multi-threaded mode, but writing to stdin/output in parallel will result in message corruption.
The easiest way to deal with this problem is to process requests sequentially.

In this case, the easiest way is to use `AsyncSemaphore`, as [described in the official guidelines](https://github.com/microsoft/vs-streamjsonrpc/blob/master/doc/resiliency.md).

In the case of plugins, performance issues related to rpc calls are rare, so we decided to limit writes by acquiring the semaphore for all methods.

### 3. Automatic generation of methods

The c-lightning plugin must expose two RPC methods: `getmanifest` and `init`.

The former is to pass information from the plugin to c-lightning, and the latter is to pass information other way around when c-lightning is launched.

Much of the processing logic can be generated automatically from other plugin method definitions, so we use reflection to keep things generic enough.

For example, `getmanifest` looks like this
```csharp
    [JsonRpcMethod("getmanifest")]
    public async Task<ManifestDto> GetManifestAsync(bool allowDeprecatedApis = false, object? otherParams = null)
    {
      using var releaser = await _semaphore.EnterAsync();
      var userDefinedMethodInfo =
        this
          .GetType()
          .GetMethods(BindingFlags.Public | BindingFlags.Instance | BindingFlags.DeclaredOnly)
          .Where(m => !m.IsSpecialName && !String.Equals(m.Name, "initasync", StringComparison.OrdinalIgnoreCase) && !String.Equals(m.Name, "getmanifestasync", StringComparison.OrdinalIgnoreCase));
      var methods =
        userDefinedMethodInfo
          .Select(m =>
          {
            var argSpec = m.GetParameters();
            var numDefaults =
              argSpec.Count(s => s.HasDefaultValue);
            var keywordArgsStartIndex = argSpec.Length - numDefaults;
            var args =
              argSpec.Where(s =>
                !string.Equals(s.Name, "plugin", StringComparison.OrdinalIgnoreCase) &&
                !string.Equals(s.Name, "request", StringComparison.OrdinalIgnoreCase))
                .Select((s, i) => i < keywordArgsStartIndex ? s.Name : $"[{s.Name}]");

            var name = m.Name.ToLowerInvariant();
            if (name.EndsWith("async"))
              name = name[..^5];
            return new RPCMethodDTO
            {
              Name = name,
              Usage = string.Join(' ', args),
              Description = _rpcDescriptions[name].Item1,
              LongDescription = _rpcDescriptions[name].Item2
            };
          });

      return new ManifestDto
      {
        Options =
          // translate `System.CommandLine` to c-lightning compatible style.
          CommandLines.GetOptions().Select(CliOptionsToDto).ToArray(),
        RpcMethods = methods.ToArray(),
        Notifications = new NotificationsDTO[]{},
        Subscriptions = new string[]{},
        Hooks = new object[] {},
        Dynamic = true,
        FeatureBits = null
      };
```

init is as follows. This is more difficult to automate than getmanifest, and the process varies depending on the requirements of the application.

```csharp
    [JsonRpcMethod("init")]
    public async Task InitAsync(LnInitConfigurationDTO configuration, Dictionary<string, object> options)
    {
      using var releaser = await _semaphore.EnterAsync();
      foreach (var op in options)
      {
        var maybeProp =
          _opts.GetType().GetProperties()
            .FirstOrDefault(p => string.Equals(p.Name, op.Key, StringComparison.OrdinalIgnoreCase));
        maybeProp?.SetValue(_opts, op.Value);
      }
      _opts.IsInitiated = true;
    }
```

init passes method arguments as objects.
The object has fields named `configurations` and `options`, So the method handler on the C# side must have a same name to automatically bind.

In this case, we only bind the startup options givin by the c-lightning side to `_opts`.
`LnInitConfigurationDto`'s information on the c-lightning side (e.g. is an user using tor? What lightning feature bits are supported? ) can be used to freely perform the initialization process.

We set `IsInitiated` at the completion of the initialization process.
This is used later to suspend the server startup process until the init message is received.

### 4. JsonRPCLogger

The fact that c-lightning and the plugin communicate via standard input/output means that logs cannot be sent to standard output at will.

The existing c-lightning plugin sends a message to the c-lightning's `log` rpc method as an rpc notification and leaves the rest of the processing to the c-lightning side.
In this way, the logs are unified on the c-lightning side, making them easier to read.

.NET is bundled with `ConsoleLogger`, but this performs the writing process as a background task, so the aforementioned thread race may occur if we implement the logging logic in the same way.

To solve these problem, [I have created my own ILoggerProvider](https://github.com/joemphilips/HelloWorldPlugin/blob/master/HelloWorldPlugin.Server/PluginLogger.cs).

### 5. startup process

c-lightning starts the plugin specified by the user at startup, passing the environment variable `LIGHTNINGD_PLUGIN=1` when launching.

Thus, the presence or absence of this environment variable can be used to determine in the program whether the application should be run as a plugin or not.

In my case, I refer to this variable in two places
The first is where you configure the host.
```csharp
Action<IHostBuilder> configureHostBuilder = hostBuilder =>
{
  var isPluginMode = Environment.GetEnvironmentVariable("LIGHTNINGD_PLUGIN") == "1";

  if (isPluginMode)
  {
    hostBuilder
      .ConfigureLogging(builder => { builder.AddJsonRpcNotificationLogger();})
      .ConfigureServices(serviceCollection =>
        serviceCollection
          .AddSingleton<MyAwesomeRpcServerOptions>()
          .AddSingleton<MyAwesomeRpcServer>()
      );
  }
  else
  {
     // configuration for running as a normal webserver.
  }
};
```

The other is during Host execution.
We delay the execution of `IHost.RunAsync()` as follows.

```csharp
  var host = hostBuilder.Build();
  var isPluginMode = Environment.GetEnvironmentVariable("LIGHTNINGD_PLUGIN") == "1";
  if (isPluginMode)
    await host.StartJsonRpcServerForInitAsync(); /// initialize only rpc server and listen to `init` call.
  await host.RunAsync();
```

This will delay the startup of `BackgroundService` and log output until the init message is received from lightningd.

### 6. compile as a single binary

Finally, the plugin must be compiled as an executable single binary.
Therefore, we utilize the [Single file Executable Compilation](https://docs.microsoft.com/en-us/dotnet/core/deploying/single-file/overview) feature of .NET

And that's it!
You can specify the `--plugin` or `--plugin-dir` of c-lightning to launch the plugin.

### publish as a library?

At first, I thought about converting what I have done this time into a library and releasing it to the public, but since it is not that hard to do it, and the init process is difficult to automate, I will just leave the document and examples here.

If you want to make it into a library, probably the best way is to use the [Source Generator](https://docs.microsoft.com/en-us/dotnet/csharp/roslyn-sdk/source-generators-overview)