Debugging C/C++ AWS IoT Greengrass Components using VSCode
Tags: iot, greengrass
Published: 2022-11-02 08:49:53


In most scenarios, simply logging the output of the application is enough to understand what is happening and perform debugging in case there is an application misbehaviour. However sometimes in order to catch some stubborn bugs stepping through the code while application is executing might be the most efficient way to do it. AWS IoT Greengrass provides a CLI tool that allows local components to be deployed and tested by also accessing the application logs, but in the scenario where we want to step through the code and understand the stack and other parts of the memory of a C/C++ component than we would need some additional configuration, which we will cover in this post.

## Prerequisites

For this setup I will be using an EC2 instance running Ubuntu 22.04 and having tools like `gdb`, `gdbserver`,  `cmake`  and `build-essential` installed as well as having my VSCode configured for [Remote Development using SSH](https://code.visualstudio.com/docs/remote/ssh). For installing AWS IoT Greengrass, I’ll be using an installation instructions provided by a [Getting Started guide](https://docs.aws.amazon.com/greengrass/v2/developerguide/getting-started.html), but any variation of this that provides the tools mentioned above, as well as AWS IoT Greengrass and Greengrass CLI installed and running, should work.


>Note that for installing Greengrass CLI together with Greengrass, supply the installer with the parameter `--deploy-dev-tools true ` which will add the Greengrass CLI component.



## Building and Running the C++ Application

Before we can build our C++ we need to make sure the we have the AWS IoT Device SDK for C++ v2 build and available to link against as it provides the Greengrass IPC library. 

```
# Create a workspace directory to hold all the SDK files
mkdir sdk-workspace
cd sdk-workspace
# Clone the repository
git clone --recursive https://github.com/aws/aws-iot-device-sdk-cpp-v2.git
# Ensure all submodules are properly updated
cd aws-iot-device-sdk-cpp-v2
git submodule update --init --recursive
cd ..
# Make a build directory for the SDK. Can use any name.
# If working with multiple SDKs, using a SDK-specific name is helpful.
mkdir aws-iot-device-sdk-cpp-v2-build
cd aws-iot-device-sdk-cpp-v2-build
# Generate the SDK build files.
# -DCMAKE_INSTALL_PREFIX needs to be the absolute/full path to the directory.
#     (Example: "/home/ubuntu/sdk-workspace/aws-iot-device-sdk-cpp-v2-build).
cmake -DCMAKE_INSTALL_PREFIX="/home/ubuntu/sdk-workspace/aws-iot-device-sdk-cpp-v2-build" ../aws-iot-device-sdk-cpp-v2
# Build and install the library. Once installed, you can develop with the SDK and run the samples
# -config can be "Release", "RelWithDebInfo", or "Debug"
cmake --build . --target install --config "Debug"
```

If everything builds successfully we can now go back and create our application directory and a C++ example:

```
cd ../../
mkdir hello-world
cd hello-world
```

In here let’s create a simple `CMakeLists.txt` file:

```
cmake_minimum_required(VERSION 3.1)
project (hello-world)

file(GLOB MAIN_SRC
        "*.h"
        "*.cpp"
        )
add_executable(${PROJECT_NAME} ${MAIN_SRC})

set_target_properties(${PROJECT_NAME} PROPERTIES
        LINKER_LANGUAGE CXX
        CXX_STANDARD 11)

find_package(aws-crt-cpp PATHS ~/sdk-workspace/aws-iot-device-sdk-cpp-v2-build)
find_package(EventstreamRpc-cpp PATHS ~/sdk-workspace/aws-iot-device-sdk-cpp-v2-build)
find_package(GreengrassIpc-cpp PATHS ~/sdk-workspace/aws-iot-device-sdk-cpp-v2-build)
target_link_libraries(${PROJECT_NAME} AWS::GreengrassIpc-cpp)
```

Finally let’s create our c++ example file which will subscribe to a specific MQTT topic (test/topic/cpp) and print out received messages `main.cpp`:

```
#include <iostream>
#include <thread>

#include <aws/crt/Api.h>
#include <aws/greengrass/GreengrassCoreIpcClient.h>

using namespace Aws::Crt;
using namespace Aws::Greengrass;

class IoTCoreResponseHandler : public SubscribeToIoTCoreStreamHandler {

    public:
        virtual ~IoTCoreResponseHandler() {}

    private:

        void OnStreamEvent(IoTCoreMessage *response) override {
            auto message = response->GetMessage();
            if (message.has_value() && message.value().GetPayload().has_value()) {
                auto messageBytes = message.value().GetPayload().value();
                std::string messageString(messageBytes.begin(), messageBytes.end());
                std::string messageTopic = message.value().GetTopicName().value().c_str();
                std::cout << "Received new message on topic: " << messageTopic << std::endl;
                std::cout << "Message: " << messageString << std::endl;
            }
        }

        bool OnStreamError(OperationError *error) override {
            std::cout << "Received an operation error: ";
            if (error->GetMessage().has_value()) {
                std::cout << error->GetMessage().value();
            }
            std::cout << std::endl;
            return false; // Return true to close stream, false to keep stream open.
        }

        void OnStreamClosed() override {
            std::cout << "Subscribe to IoT Core stream closed." << std::endl;
        }
};

class IpcClientLifecycleHandler : public ConnectionLifecycleHandler {
    void OnConnectCallback() override {
        std::cout << "OnConnectCallback" << std::endl;
    }

    void OnDisconnectCallback(RpcError error) override {
        std::cout << "OnDisconnectCallback: " << error.StatusToString() << std::endl;
        exit(-1);
    }

    bool OnErrorCallback(RpcError error) override {
        std::cout << "OnErrorCallback: " << error.StatusToString() << std::endl;
        return true;
    }
};

int main() {
    String topic("test/topic/cpp");
    QOS qos = QOS_AT_LEAST_ONCE;
    int timeout = 10;

    ApiHandle apiHandle(g_allocator);
    Io::EventLoopGroup eventLoopGroup(1);
    Io::DefaultHostResolver socketResolver(eventLoopGroup, 64, 30);
    Io::ClientBootstrap bootstrap(eventLoopGroup, socketResolver);
    IpcClientLifecycleHandler ipcLifecycleHandler;
    GreengrassCoreIpcClient ipcClient(bootstrap);
    auto connectionStatus = ipcClient.Connect(ipcLifecycleHandler).get();
    if (!connectionStatus) {
        std::cerr << "Failed to establish IPC connection: " << connectionStatus.StatusToString() << std::endl;
        exit(-1);
    }

    SubscribeToIoTCoreRequest request;
    request.SetTopicName(topic);
    request.SetQos(qos);
    auto streamHandler = MakeShared<IoTCoreResponseHandler>(DefaultAllocator());
    auto operation = ipcClient.NewSubscribeToIoTCore(streamHandler);
    auto activate = operation->Activate(request, nullptr);
    activate.wait();

    auto responseFuture = operation->GetResult();
    if (responseFuture.wait_for(std::chrono::seconds(timeout)) == std::future_status::timeout) {
        std::cerr << "Operation timed out while waiting for response from Greengrass Core." << std::endl;
        exit(-1);
    }

    auto response = responseFuture.get();
    if (response) {
        std::cout << "Successfully subscribed to topic: " << topic << std::endl;
    } else {
        // An error occurred.
        std::cout << "Failed to subscribe to topic: " << topic << std::endl;
        auto errorType = response.GetResultType();
        if (errorType == OPERATION_ERROR) {
            auto *error = response.GetOperationError();
            std::cout << "Operation error: " << error->GetMessage().value() << std::endl;
        } else {
            std::cout << "RPC error: " << response.GetRpcError() << std::endl;
        }
        exit(-1);
    }

    // Keep the main thread alive, or the process will exit.
    while (true) {
        std::this_thread::sleep_for(std::chrono::seconds(10));
    }

    operation->Close();
    return 0;
}
```

At this point we should be able to build the application:

```
mkdir build
cd build
cmake -DCMAKE_PREFIX_PATH="/home/ubuntu/sdk-workspace/aws-iot-device-sdk-cpp-v2-build" -DCMAKE_BUILD_TYPE="Debug" ..
cmake --build . --config "Debug"
```

Since this can work only in the context of Greengrass we need to prepare the component yaml file as well as the artifact:

```
cd ..
mkdir -p gg/artifacts/com.example.HelloWorld/1.0.0
mkdir -p gg/recipes
touch gg/recipes/com.example.HelloWorld-1.0.0.yaml
```

Where the content of the `com.example.HelloWorld-1.0.0.yaml` would be:

```
RecipeFormatVersion: '2020-01-25'
ComponentName: com.example.HelloWorld
ComponentVersion: 1.0.0
ComponentDescription: My C++ component.
ComponentConfiguration:
  DefaultConfiguration:
    accessControl:
      aws.greengrass.ipc.mqttproxy:
        com.example.HelloWorld:mqttproxy:1:
          policyDescription: Allows access to subscribe to a topics.
          operations:
            - aws.greengrass#SubscribeToIoTCore
          resources:
            - "test/topic/cpp"
Manifests:
- Platform:
    os: linux
  Lifecycle:
    Run: "{artifacts:path}/hello-world"
```

Now we can copy the `hello-world` binary to artifacts directory and deploy the component to Greengrass:

```
cp build/hello-world gg/artifacts/com.example.HelloWorld/1.0.0/

sudo /greengrass/v2/bin/greengrass-cli deployment create \
  --recipeDir gg/recipes \
  --artifactDir gg/artifacts \
  --merge "com.example.HelloWorld=1.0.0"
```

After this if look at the logs, we should see:

```
sudo bash -c "cat /greengrass/v2/logs/com.example.HelloWorld.log"
...
com.example.HelloWorld: stdout. Successfully subscribed to topic: test/topic/cpp. {scriptName=services.com.example.HelloWorld.lifecycle.Run, serviceName=com.example.HelloWorld, currentState=RUNNING}
```

Once we verify that this is working properly, we can jump to the part where we actually do a step through debugging.

## Debugging using GDB and VSCode

First let’s set up our `.vscode/launch.json` to use the `gdbserver` 

```
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "HelloWorld GG Component",
            "type": "cppdbg",
            "request": "launch",
            "program": "${workspaceFolder}/hello-world/gg/artifacts/com.example.HelloWorld/1.0.0/hello-world",
            "miDebuggerServerAddress": "localhost:9091",
            "args": [],
            "stopAtEntry": true,
            "cwd": "${workspaceRoot}",
            "environment": [],
            "externalConsole": false,
            "serverStarted": "Listening on port",
            "filterStderr": true,
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "enable-pretty-printing",
                    "ignoreFailures": true,
                }
            ],
            "MIMode": "gdb",
        }
    ]
}
```

While this is a typical configuration two things we need to make sure are set right. 

1. `program` is pointing to where the actual artifact is being installed by the Greengrass CLI, 
2. `miDebuggerServerAddress` is set to right host and port. In this scenario since we are doing this locally we will be using the `localhost` and port of choice is `9091` which we need to match when modifying the Greengrass component recipe.

Next we will instruct Greengrass to start the `gdbserver` when starting the component, which will allow us to use `gdb` for remote debugging.

```
RecipeFormatVersion: '2020-01-25'
ComponentName: com.example.HelloWorld
ComponentVersion: 1.0.0
ComponentDescription: My C++ component.
ComponentConfiguration:
  DefaultConfiguration:
    accessControl:
      aws.greengrass.ipc.mqttproxy:
        com.example.HelloWorld:mqttproxy:1:
          policyDescription: Allows access to subscribe to a topics.
          operations:
            - aws.greengrass#SubscribeToIoTCore
          resources:
            - "test/topic/cpp"
Manifests:
- Platform:
    os: linux
  Lifecycle:
    Run: "gdbserver :9091 {artifacts:path}/hello-world"
```

Here the only difference is the `Run` command which we prefixed with `gdbserver :9091`. After this is done we can redeploy the component:

```
sudo /greengrass/v2/bin/greengrass-cli deployment create \
  --recipeDir gg/recipes \
  --artifactDir gg/artifacts \
  --merge "com.example.HelloWorld=1.0.0"
```

After which we should see the following in the component logs:

```
sudo bash -c "cat /greengrass/v2/logs/com.example.HelloWorld.log"
com.example.HelloWorld: stderr. Listening on port 9091. {scriptName=services.com.example.HelloWorld.lifecycle.Run, serviceName=com.example.HelloWorld, currentState=RUNNING}
```

At this point, we can just create a breakpoint and start the debugging session:
![VSCode Debug Main](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p62u5r9tulusgb7fx1um.png)
We can then put a break point at line 19, which we will let us stop the application upon a retrieval of a message from the AWS IoT Core. 
In order to test this we can go to AWS console → AWS IoT Core → MQTT test client → Publish to a topic and publish a message to the `test/topic/cpp` like the example bellow.
![AWS IoT MQTT Client Test](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x8r902e9ptyu5p1qn7tg.png)
We will be able to catch this with the breakpoint and step through if necessary.
![VSCode Debug](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cc6pxu52xunbyt9t3duq.png)
Additionally after the message got received and processed we will see it in the component log as well.

## Conclusion

This setup allows us to use `gdbserver` / `gdb` and VSCode to visualise and inspect what is happening in our GGv2 components. We can go further and even debug multiple components at the same time, where the only thing we need to change would be the `gdbserver` ports on which those applications are running and modify it in the recipe respectively.

If you find this interesting or have suggestions for future topics feel free to reach out here or @nenadilc84 on [Twitter](https://twitter.com/nenadilic84) or [LinkedIn](https://www.linkedin.com/in/nenadilic84/).

> There is also a [video](https://youtu.be/qMNZJ0CHZ9U) version of this blog