Using VRChat OSC to control VRCHAT with a MIDI device

# Objective
Use MIDI Device(Such as MIDI Keyboard,Pedal) as VRCHAT motion control function(OSC)

# Join VRChat Beta program
Open Steam and open VRCHAT property window.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/a58342e4-c3e7-09ad-d0c0-abf5e8480c81.png)

Choice `open-beta - Open Beta`
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/49feeb86-3d44-e3e9-468a-e7de2ac2035d.png)
Close window.
It should be reloading.

# Check VRCHAT+OSC is enabled.
Launch VRCHAT at desktop mode.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/e547cbd1-4cce-f3d2-1513-785d78e0a32f.png)

Press R hold and click options.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/40041b15-1b56-f2b9-0384-221cd159f7ec.png)
Make sure you can see OSC menu.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/0d824320-483a-0399-6aab-e1838c2bbb7d.png)
Click OSC Debug and check it.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/3690cf52-d0a3-7d02-1044-fd5571213ba0.png)
It is OK to proceed next step if you can see this message.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/20e756a9-9aea-a1a9-d4a0-b084675de833.png)

If you see `OSC was turned off in your settings ~~~` at first line....
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/0578533e-b609-c8f8-9003-1c071fc58b38.png)
Open OSC menu and make sure OSC Enabled is enabled.(Picture's status is disabled)
Just click it and enable it.

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/e58d448c-00e1-ebd5-3a8f-8fe51e15a83e.png)

Nice! Let's go next step, DO NOT close your VRCHAT and keep the debug screen up..

# Install TouchOSC
https://hexler.net/touchosc#_
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/24131d4f-c923-1f75-711e-bfae0d94cd8c.png)
Click `GET TOUCHOSC`
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/3b9d1efe-0ca1-9db8-8542-e6f9e3d53f7d.png)
Click `Windows` and Download it

## Launch TouchOSC
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/d7844776-ad62-8458-6024-91ffc95bea32.png)

If you have money, Why don't you purchase licences?

## Connect settings TouchOSC to VRCHAT
Choice TouchOSC　`Edit`→`Connections`
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/2412a4a3-9143-fe2a-94a5-e16fe148b8ed.png)

Configure like screenshot as below(FYI, It will NOT work if you set host to "localhost" set "127.0.0.1")
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/8ec8891b-7689-7500-80f6-7e5fa422eeb1.png)
Done and back to main window.

## Add control button
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/09394155-97b7-82f6-8e4a-01948f438876.png)

As a test, add button.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/be771630-f469-2a3a-5132-3ade7baed558.png)

Create button
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/b56247fc-631f-1d18-ead5-e0ea729e6bcf.png)

## Test sending click event from TouchOSC 
Maybe you can check more easily if you put a window side by side.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/26bc3e1d-ac7e-8d4e-c045-182da6691b10.png)

Click Run button(▶) in TouchOSC
![Animation.gif](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/73aea8c6-b5a1-5784-4ad2-ea0c473715b1.gif)
You can check that TouchOSC sent button event from OSC to VRCHAT

If you want to close run mode, click a point which locate upright.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/b9a3a60b-6f70-b6b5-9b09-341db7bedd87.png)


# [Main topic] OSC events triggered by MIDI signals
## Configure MIDI input in TouchOSC
TouchOSC `Edit`→`Connections`→`MIDI`
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/0a2cbf63-9709-fe33-9d7a-cb264b410017.png)
Click "Browse" and set MIDI Device output(Send Port)and input(Receive Port) 
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/a51bacad-1432-3865-c5b9-3174c4359c0c.png)
Done!


## Install MIDI monitoring tool "Protokol"
https://hexler.net/protokol
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/d1b537e2-5ce3-e090-7e26-15aa67e975dd.png)

##  Monitor MIDI note data
MIDI note(ex. C1 at keyboard)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/0fd37678-e359-9580-863b-e618e018e71f.png)
MIDI Control Change(ex. sustain pedal as 64)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/7326477e-93f9-2e07-230b-0dbe4b0cbc50.png)

`RECEIVE    | ENDPOINT(A-Series Keyboard) TYPE(CONTROLCHANGE) CHANNEL(1) DATA1(64) DATA2(79)`
は
`RECEIVE | MIDI (DEVICENAME) TYPE(TYPE NAME) CHANNEL(1) DATA1(ControllNo.:64) DATA2(Velocity:0-127)

These numbers and parameters will be use in MIDI input mapping phase.

## Make MIDI input to OSC event
Overall GIF
![Animation.gif](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/3620e7d8-3124-4004-dae7-f80957d3299e.gif)

- Click "Button" and show details.
- Expand `Messages`→`MIDI` tab details. 

![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/fd36cc7a-ebeb-860b-f165-c3dadf34ca0e.png)
Enable  →Check
Send    → Not Check
Receive →Check
Feedback→Check
Connections: 1 OK　MIDI input device No.
Trigger : Default OK(as screenshot shows)
Type: if use midi note `NOTE_ON`, if you use pedals,`CONTROL CHANGE`
Channel: Choice CONSTANT input channel No. which you checked at monitar(here:1)  
Note: Choice CONSTANT input midi note No.(Here, C1key :64)
Velocity: Default OK(x : VALUE)
Scale: Set input range.(0-127)

## Test with VRCHAT OSC DEBUG 
Click TouchOSC Run(▶) button.
Animation show 
In the following animation, Protokol is shown in the lower right to see the MIDI response, VRCHAT's ORC DEBUG screen is running in the background, and OSC DEBUG is running in the lower left.
Here's how it works when I press the C1 (Do) key as a test!
![Animation.gif](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/686b23ca-95c7-476b-18ac-372022b052c6.gif)



## Assign to VRCHAT motion
Now that we have successfully sent the data to VRCHAT, let's assign the VRCHAT functions.`C:\Users\{$USERNAME}\AppData\LocalLow\VRChat\VRChat\OSC\{hogehoge}\Avatars`
There is a json for each avatar

This JSON has a parameter called input, and by sending a request with the specified type (Int in this case) in this address, the corresponding function will become available.
By setting the address listed in `address` as the destination of TouchOSC button events, you can use the avatar function from OSC. 
```avatername.json
    {
      "name": "VRCEmote",
      "input": {
        "address": "/avatar/parameters/VRCEmote",
        "type": "Int"
      },
      "output": {
        "address": "/avatar/parameters/VRCEmote",
        "type": "Int"
      }
    },
```
This time we will use the default Emote function of VRC (`avatar/parameters/VRCEmote`). (It's like Wave or Die...).
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/9b7da96a-8e0f-69ec-f63e-dad3358ace1d.png)

Click on a TouchOSC button and change the value of `Control`→`Name` from the default value of button1 in Detail View.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/0c029d61-82c5-9a33-8749-b37585340d8f.png)
→The first `/` should be pulled out and replaced with `→avatar/parameters/VRCEmote`.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/067b10d2-acaf-aba0-9ecb-f6f6ac1286ef.png)

After making the changes, run TouchOSC again (▶) and press the C1 (Do) key.
![Animation.gif](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/78835bee-64cf-7b7d-75f5-692cf7607e3a.gif)



A single parameter may have multiple functions assigned to it. For example, in `avatar/parameters/VRCEmote`, you can use other functions such as Clap and Die depending on the Int value you give.
To change the Int value, click the TouchOSC `button` -> `right side detail view` -> `Messages` -> `OSC` -> change the `Scale` range.
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/37fe20fe-6707-aeba-8806-a985ca89ccf5.png)
If we set this to 2, we get ....
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/22b2a5a0-4180-36c7-942b-b420d95f02ac.png)

![Animation.gif](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/171398/db2d61a9-88a4-4ff2-3c70-1ace77108333.gif)


# In closing
I've been using VRCHAT for about two weeks now, and I'd like to thank the community for being so kind to me, and I hope this article will help those of you who are trying to enjoy VRCHAT more.

## Reference
https://note.com/bd_/n/n4279a0c08b28

https://hexler.net/touchosc/manual/editor-messages-midi#connections


## Special Thanks
@dokkozo He provided information on the section to assign to VRCHAT motions.
Also, his(her) avatars are super very kawaii, which motivates me.