Tutorial: Hacking a robot while respecting fundamental laws of robotics
Tags: robots, javascript, tutorial, programming
Published: 2019-08-19 18:43:45

---
title: Tutorial: Hacking a robot while respecting fundamental laws of robotics
published: true
description: This year at Twilio Signal conference, I got to play with Misty robots 🤖. Read more on how to control a robot with a few API calls.
tags: robots, javascript, tutorial, programming
cover_image: https://www.mistyrobotics.com/wp-content/uploads/DevDeskMistyOnly.jpg
---

This past week I got invited to attend [Twilio Signal](https://signal.twilio.com) conference in San Francisco as part of their Champions program.

I had the chance to meet other awesome Champions at our Summit. You should all follow them, they are all doing amazing things in their communities all around the world. Find more about the Twilio Champions program [over here](https://www.twilio.com/champions).

The conference itself was full of great announcements around the Twilio eco-system. I am mostly excited by [Conversations](https://www.twilio.com/conversations), as a way to connect many people uing different systems (SMS, WhatsApp, chat...) into the same conversation. The new [Twilio CLI](https://github.com/twilio/twilio-cli) to do everything around Twilio directly in the terminal looks pretty awesome too.

Another big announcement at Signal was version 3 of [TwilioQuest](https://www.twilio.com/quest) available on Desktop. If you are not familiar with TwilioQuest, imagine a video game where you gain XP points by solving coding challenges.
It's a fun way to learn about the Twilio world and programming in general.

![screenshot of TwilioQuest](https://www.twilio.com/quest/next/img/screenshots/explore.png)
<figcaption>Screenshot of TwilioQuest</figcaption>

The main character you interact within TwilioQuest, is named *Cedric* and is a friendly robot that is guiding you through your quest against the *Legacy Systems*.

The thing I did not realize: Cedric is real! He and his friends attended Signal too. They were so many roaming the expo floor and booths.

There even was a special Signal mission on TwilioQuest to earn XPs if you were spending some time with the team of [Misty Robotics](https://mistyrobotics.com) (parent company of the Misty robot, Cedric's robot family).

And this exactly what I did! I went over to meet the Misty Robotics team to learn more about those friendly robots🤖

The team was organizing Developers testing sessions, and inviting people to try their *Hello World* tutorial to "hack" the robot.

Misty is an incredible robot, among many things she has multiple sensors to move around your house without bumping into anything, a speaker to express herself, a screen to show some emotions, and a camera with face recognition capability. And all that is hackable through an SDK! 🎉

![Misty robot look](https://thepracticaldev.s3.amazonaws.com/i/pxgaixc6e8stohh53h3o.jpg)
<figcaption>Isn't she lovely?</figcaption>

I had a lot of fun following the tutorial. I made Misty move, make her blink her light and even taught her to recognize me. At the end of the session, the team was handing out surveys to collect feedback. And as you may have already guessed it was a Typeform! 🤩

Then I had no option... I had to connect Misty to Typeform.

I wanted to make Misty react every time someone was filling up the survey.

Here is how I managed to connect the two.

# Overall principle 📖
We rely on Typeform webhooks functionality to react immediately after someone answered.

The webhook is received by an app that can call the Misty API and make the robot react.

But to be able to communicate with the Misty robot, your app needs to be on the same WIFI network, so it's most likely that you will run it locally on your laptop.

We will use [ngrok](http://ngrok.io) to expose localhost to the world 🌍.

# Make the Misty move her arm
We create a small express app with only one endpoint in a file named `app.js`. This endpoint will receive webhook notification and then call Misty API.

This is how it looks:

```javascript
const express = require('express')
const app = express()
const port = 3000

const axios = require('axios')
const bodyParser = require('body-parser')

// parse application/x-www-form-urlencoded
app.use(bodyParser.urlencoded({ extended: false }))

// parse application/json
app.use(bodyParser.json())

const ROBOT_IP = '10.41.129.96' //change to your own
app.post('/', (req, res) => {
  res.sendStatus(200)

  axios({
    method: 'POST',
    url: `http://${ROBOT_IP}/api/arms`,
    data:{
        Arm: "left",
        Position: -90,
        Velocity: 100,
        Units: "degrees"
      }
  }).catch(function(err){
    console.log(err)
  })
})

app.listen(port, () => console.log(`Example app listening on port ${port}!`))
```

You can run the app by using the command `node app.js`

And to make her arm go up, run the following command `curl -X POST https://localhost:300` in your terminal. 

And if everything works well you should see Misty's left arm going up 🎉

# Connect it to Typeform 🔗
All this is working well locally, but wouldn't it be cool to connect it to other services?

To make it happen we need the help of an awesome tool called [ngrok](https://ngrok.com/). ngrok will expose your localhost and make it available to rest of the internet by giving it a URL. This URL is unique to you and every time you launch ngrok. Make sure to read its [getting started guide](https://dashboard.ngrok.com/get-started).

Once you have ngrok installed you can run the command `ngrok http 300`.
This should give you a URL back. We are going to use it to call our app from the outside.

![Screenshot of ngrok](https://www.evernote.com/l/ACUtnS6C0VFJKLTaOVZKDdJGkKcPch-HAy4B/image.png)
<figcaption>Screenshot of ngrok</figcaption>

On your Typeform dashboard, we can now select the form we want to connect to Misty, and under _Connect > Webhooks_ add a new webhook with this URL.

And voilà 🎉
You now have a robot that raises her arm every time someone fills up your typeform.

This will work until you kill ngrok or the node app. Ngrok will give you a new URL every time you launch it, so just keep it open while you are developing.

# Go beyond 🚀
After seeing Misty raising her arm the first time, I could not restrain myself, and I shouted a loud "Woo-hoo" on the conference floor 😊.

But I could not stop jus now when I was so close to building something even cooler.

My ultimate goal was to build an app that will make Misty speak and react to the things that were posted on the typeform.

## Make her arm move dynamically 👋
So I built a [simple typeform](https://picsoung.typeform.com/to/n6NnNI), where people could leave their name and decide which arm Misty should raise. 

To make it easier to extract data from typeform webhook payload, I had to update the `ref` of my questions. On your typeform select the question and look at the bottom of the sidebar and click `Edit`. There you can change it to whatever you please. I called mine `username` and `arm_choice`.

![screenshot typeform edit ref](https://d1nt04ao0vck22.cloudfront.net/uploads/2015/07/09121737/question-settings-2.png)
<figcaption>Edit Ref on typeform question</figcaption>

Now let's see how to extract those values from the webhook payload 👇

```javascript
  const { form_response } = req.body
  const {answers} = form_response
 
  let arm_field = answers.find((a) => a.field.ref === 'arm_choice')
  let selected_arm = arm_field.choice.label

  let username_field = answers.find((a) => a.field.ref === 'username')
  let username = username_field.text
```

Then you can pass `select_arm` value to Misty API, and raise the corresponding arm.

```javascript
axios({
    method: 'POST',
    url: `http://${ROBOT_IP}/api/arms`,
    data:{
        Arm: selected_arm,
        Position: -90,
        Velocity: 100,
        Units: "degrees"
      }
  }).catch(function(err){
    console.log(err)
  })
```

## Maker her speak 🤖🗣️
Misty REST API lets you send WAV audio files using the `SaveAudio` endpoint ([doc](https://docs.mistyrobotics.com/misty-ii/reference/rest/#saveaudio-data-string-)) but... they have to be encoded in base64 😤

So I had to find a hack to generate a `.wav` file and convert it to base64.

It took me a couple of trials to find the perfect library to do it, but [say.js](https://github.com/Marak/say.js/) saved my day!

```javascript
const say = require('say')
say.export(`Hello ${username}`, 'Alex', 0.75, `hello_${username}.wav`, (err) => {
  if (err) {
    return console.error(err)
  }
})
```

Using those few lines, it creates a `.wav` file named `hello_${username}.wave` with a voice that says `Hello nicolas` for example.
Unfortunately, I could not find the name for the female voices on my mac, so we are stuck with Alex for now 🤷‍♂️

```javascript
cont fs = require('fs');
let wav = new WaveFile(fs.readFileSync(`hello_${username}.wav`));
let wav64 = wav.toBase64()

  axios({
    method: 'POST',
    url: `http://${ROBOT_IP}/api/audio`,
    data:{
      FileName: `hello_${username}.wav`,
      Data: wav64,
      ImmediatelyApply: true, //make Misty play it right away
      OverwriteExisting: true
    }
  }).catch(function(err){
    console.log(err)
  })
```

And this 🔝 is how I read the file, convert it to base64 and call Misty API to make it play on the robot.

And this how it finally looks 🤩
{% twitter 1159195895646183424 %}

The code for the entire app is available here👇

{% gist https://gist.github.com/picsoung/d3cd15ea6bd98e5e1f8c669308119ef2 %}

Hope that got you excited to hack things around and play with Misty 🤖.
Their crowdfunding campaign is over, but you can already [pre-order](https://shop.mistyrobotics.com/) it.


🙏 Special thanks to Twilio Champions team for inviting me to be part of this great adventure 😃