[.NET 6][Uno Platform][Ubuntu 21.10] Using Leap Motion
Tags: dotnet, csharp, leapmotion, ubuntu
Published: 2021-12-29 16:47:51

## Intro
In this time, I will try using Leap Motion Controller from the Uno Platform application on Ubuntu.

* [Ultraleap for Developers](https://developer.leapmotion.com/)

I use this project to add LeapSDK.

* [[C#][Uno Platform] Create desktop applications on Ubuntu 1](https://dev.to/masanori_msl/cuno-platform-create-desktop-applications-on-ubuntu-1-3o3i)

### Environments

* Ubuntu 21.10
* .NET 6.0.101
* LeapDeveloperKit ver.2.3.1

## Using Leap Motion Controller on Ubuntu

To use Leap Motion Controller on Ubuntu I have to install tracking software.

* [Download Ultraleap's Hand Tracking Software - Ultraleap for Developers](https://developer.leapmotion.com/tracking-software-download)

After unpacking the tgz file, install a deb file.

```
sudo dpkg --install Leap-2.3.1+31549-x64.deb
```

And I also install Mesa OpenGL.

```
sudo apt-get install libgl1-mesa-dri libgl1-mesa-glx
```

After installing, I start the daemon of Leap Motion.

```
sudo leapd
```

I can open a control panel.

```
LeapControlPanel &
```

To open the control panel, I have to execute the command twice.
I don't know why :(

* [Linux Installation - Ultraleap Tracking Support](https://support.leapmotion.com/hc/en-us/articles/360004317918-Linux-Installation)

### Controller was not working

Though I didn't get any errors, my machine hasn't recognized the controller yet. 

According to this post, I could know I had to add a config file(leapd.service).

* [忙中偸閑=망중투한: [Note] Leap Motion on Ubuntu 20.04](https://blog.keithkim.com/2020/07/note-leap-motion-on-ubuntu-2004.html)

Now I can use the Leap Motion Controller on the control panel.

## Add a LeapSDK library file in .NET 6 projects

In the tgz file, I can also get library files to control the Leap Motion Controller.
So I add a dll file for .NET Framework 4.0.

#### UnoSample.Skia.Gtk.csproj
```xml
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType Condition="'$(Configuration)'=='Release'">WinExe</OutputType>
    <OutputType Condition="'$(Configuration)'=='Debug'">Exe</OutputType>
    <TargetFramework>net6.0</TargetFramework>
    <LangVersion>latest</LangVersion>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>
  <ItemGroup Condition="exists('..\UnoSample.UWP')">
    <EmbeddedResource Include="..\UnoSample.UWP\Package.appxmanifest" LogicalName="Package.appxmanifest" />
    <Content Include="..\UnoSample.UWP\Assets\StoreLogo.png" Link="Assets\StoreLogo.png" />
    <Content Include="Assets\Fonts\uno-fluentui-assets.ttf" />
  </ItemGroup>
  <ItemGroup>
    <UpToDateCheckInput Include="..\UnoSample.Shared\**\*.xaml" />
  </ItemGroup>
  <ItemGroup>
    <Reference Include="Leap">
      <HintPath>References/LeapSdk/LeapCSharp.NET4.0.dll</HintPath>
    </Reference>
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.Logging" Version="5.0.0" />
    <PackageReference Include="Microsoft.Extensions.Logging.Console" Version="5.0.0" />
    <PackageReference Include="Uno.UI.Skia.Gtk" Version="4.0.11" />
    <PackageReference Include="Uno.UI.RemoteControl" Version="4.0.11" Condition="'$(Configuration)'=='Debug'" />
    <PackageReference Include="Uno.UI.Adapter.Microsoft.Extensions.Logging" Version="4.0.11" />
  </ItemGroup>
  <Import Project="..\UnoSample.Shared\UnoSample.Shared.projitems" Label="Shared" />
</Project>
```

* [【.NET】Add files into projects](https://dev.to/masanori_msl/net-add-files-into-projects-5en)

### libLeapCSharp can't be found

Though I don't get any compile errors and I can get static value like "Gesture.GestureType.TYPE_CIRCLE",
when I add classes for controlling the Leap Motion Controller, I get a runtime exception.

#### LeapCtrlListener.cs
```cs
using Leap;

namespace UnoSample.Skia.Gtk.MotionControllers
{
    public class LeapCtrlListener: Listener
    {
        public LeapCtrlListener()
        {
            Console.WriteLine("LeapCtrlListener");
        }
    }
}

```
#### Program.cs
```cs
using GLib;
using Uno.UI.Runtime.Skia;
using Leap;
using UnoSample.Skia.Gtk.MotionControllers;

namespace UnoSample.Skia.Gtk
{
	class Program
	{
		static void Main(string[] args)
		{
			// OK. "Gesture: TYPE_CIRCLE"
			Console.WriteLine("Gesture: " + Gesture.GestureType.TYPE_CIRCLE);
			
			// Error
			var leapCtrlListener = new LeapCtrlListener();
			
			ExceptionManager.UnhandledException += delegate (UnhandledExceptionArgs expArgs)
			{
				Console.WriteLine("GLIB UNHANDLED EXCEPTION" + expArgs.ExceptionObject.ToString());
				expArgs.ExitApplication = true;
			};

			var host = new GtkHost(() => new App(), args);

			host.Run();
		}
	}
}
```

#### Error
```
Unhandled exception. System.TypeInitializationException: The type initializer for 'Leap.LeapPINVOKE' threw an exception.
 ---> System.TypeInitializationException: The type initializer for 'SWIGExceptionHelper' threw an exception.
 ---> System.DllNotFoundException: Unable to load shared library 'LeapCSharp' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libLeapCSharp: cannot open shared object file: No such file or directory
   at Leap.LeapPINVOKE.SWIGExceptionHelper.SWIGRegisterExceptionCallbacks_Leap(ExceptionDelegate applicationDelegate, ExceptionDelegate arithmeticDelegate, ExceptionDelegate divideByZeroDelegate, ExceptionDelegate indexOutOfRangeDelegate, ExceptionDelegate invalidCastDelegate, ExceptionDelegate invalidOperationDelegate, ExceptionDelegate ioDelegate, ExceptionDelegate nullReferenceDelegate, ExceptionDelegate outOfMemoryDelegate, ExceptionDelegate overflowDelegate, ExceptionDelegate systemExceptionDelegate)
   at Leap.LeapPINVOKE.SWIGExceptionHelper..cctor()
   --- End of inner exception stack trace ---
   at Leap.LeapPINVOKE.SWIGExceptionHelper..ctor()
   at Leap.LeapPINVOKE..cctor()
   --- End of inner exception stack trace ---
   at Leap.LeapPINVOKE.new_Listener()
   at Leap.Listener..ctor()
   at UnoSample.Skia.Gtk.LeapMotions.LeapController..ctor() in /home/example/Documents/workspace/UnoSample/UnoSample.Skia.Gtk/LeapMotions/LeapController.cs:line 7
   at UnoSample.Skia.Gtk.Program.Main(String[] args) in /home/example/Documents/workspace/UnoSample/UnoSample.Skia.Gtk/Program.cs:line 13
```

### Add link to shared object file

To use the library file, I have to add link to "libLeapCSharp.so" on my machine.

I can execute this command before run the project.

```
export LD_LIBRARY_PATH="$Home/home/example/Documents/leap_developer_kit"
```

(In "leap_developer_kit" folder, there are "libLeapCSharp.so", "libLeap.so", and so on)


But because I don't want to do this everytime.

So I add "leap_developer_kit.conf" into "/etc/ld.so.conf.d".

#### leap_developer_kit.conf
```
/home/example/Documents/leap_developer_kit
```

After I execute "sudo ldconfig", I can watch the link of the files by "ldconfig -p".

```
...
  libLeapCSharp.so (libc6,x86-64) => /home/example/Documents/leap_developer_kit/libLeapCSharp.so
	libLeap.so (libc6,x86-64) => /home/example/Documents/leap_developer_kit/libLeap.so
...
```

* [EnvironmentVariables - Community Help Wiki](https://help.ubuntu.com/community/EnvironmentVariables)
* [(6) ldconfigでライブラリパスを更新 - Linuxやってみる！](https://www.dogrow.net/linux/blog6/)

## (Sample) Getting hand count
According to the sample code in the tgz file, I add classes and get tracking hand count.

#### Program.cs
```cs
using GLib;
using Uno.UI.Runtime.Skia;
using UnoSample.Skia.Gtk.MotionControllers;

namespace UnoSample.Skia.Gtk
{
	class Program
	{
		static void Main(string[] args)
		{
			using(var motionController = new MotionController())
			{
				ExceptionManager.UnhandledException += delegate (UnhandledExceptionArgs expArgs)
				{
					Console.WriteLine("GLIB UNHANDLED EXCEPTION" + expArgs.ExceptionObject.ToString());
					expArgs.ExitApplication = true;
				};
				var host = new GtkHost(() => new App(), args);
				host.Run();
			}
		}
	}
}
```

#### MotionController.cs
```cs
using Leap;

namespace UnoSample.Skia.Gtk.MotionControllers;
public class MotionController: IDisposable
{
    private LeapCtrlListener leapCtrlListener;
    private Controller controller;
    public MotionController()
    {
        this.leapCtrlListener = new LeapCtrlListener();
        this.controller = new Controller ();
        this.controller.AddListener(this.leapCtrlListener);
    }
    public void Dispose()
    {
        this.controller.RemoveListener(this.leapCtrlListener);
        this.controller.Dispose();
    }
}
```

#### LeapCtrlListener.cs
```cs
using Leap;

namespace UnoSample.Skia.Gtk.MotionControllers;
public class LeapCtrlListener : Listener
{
    public LeapCtrlListener()
    {
        Console.WriteLine("LeapCtrlListener");
    }
    public override void OnInit(Controller controller)
    {
        Console.WriteLine("Initialized");
    }
    public override void OnConnect(Controller controller)
    {
        Console.WriteLine("Connected");
        controller.EnableGesture(Gesture.GestureType.TYPE_CIRCLE);
        controller.EnableGesture(Gesture.GestureType.TYPE_KEY_TAP);
        controller.EnableGesture(Gesture.GestureType.TYPE_SCREEN_TAP);
        controller.EnableGesture(Gesture.GestureType.TYPE_SWIPE);
    }
    public override void OnDisconnect(Controller controller)
    {
        //Note: not dispatched when running in a debugger.
        Console.WriteLine("Disconnected");
    }
    public override void OnExit(Controller controller)
    {
        Console.WriteLine("Exited");
    }
    public override void OnFrame(Controller controller)
    {
        var leapFrame = controller.Frame();
        var handList = leapFrame.Hands;
        Console.WriteLine($"OnFrame Hand: {handList.Count}");
    }
}
```