DJI FlightHub Sync: Media File Direct Transfer to AWS
Tags: drone, dji, api, aws
Published: 2024-03-31 10:19:49

There are many drone management platforms and DJI has its own called [FlightHub 2](https://enterprise.dji.com/flighthub-2). It's quite a powerful software, where you can do a route planning, mission management, watch livestream from a drone or do some basic media files management.

Drones have many use cases, so sometimes there is a need to customize your workflow - e.g. use your own AI/ML models for object detection or share media files captured by a drone to with a 3rd party software.

To achieve this - you have to somehow integrate your software with the DJI ecosystem. There are a few possibilities how to do this.

## DJI Cloud API

There is [DJI Cloud API](https://developer.dji.com/cloud-api/), which is a powerful framework that allows developers to interact with DJI drones and their associated data through common standard protocols such as MQTT, HTTPS, and WebSocket. 

It gives you a great flexibility, but in some cases it would be too much work. The problem is that you can either use FlightHub 2 or your Cloud API based solution. Using both at the same time is not supported, see [Can the third-party platform developed by Cloud API and Flighthub 2 be used at the same time](https://sdk-forum.dji.net/hc/en-us/articles/6925988636441-Can-the-third-party-platform-developed-by-Cloud-API-and-Flighthub-2-be-used-at-the-same-time) support topic.

This means that you can't for example process media files in our own software, but do the rest e.g. route planning, mission management in FlightHub 2. You would have to also implement those advanced features in your app.

Fortunately, DJI has created a solution for such use cases - [FlightHub Sync](https://fh.dji.com/user-manual/en/flighthub-sync/introduction.html).

## FlightHub Sync

[FlightHub Sync](https://fh.dji.com/user-manual/en/flighthub-sync/introduction.html) is a feature within DJI FlightHub 2 that facilitates communication and data exchange between FlightHub 2 and third-party platforms. This includes APIs for
- Media File Direct Transfer
- Telemetry Data Transfer
- Stream Forwarding

We will focus on media files (photos, videos) transfer.

To configure FlightHub Sync, you must have "Super Admin" or "Organization Admin" role. Go to [My Organization](https://fh.dji.com/user-center#/my-organization) and click "Organization Settings" action (icon button with a "cog").

!["Organization Settings" action (icon button with a "cog")](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ki2pk0v5y3k4oav28eig.png)

You will see FlightHub Sync (Beta) configuration in the upper-right corner.

![FlightHub Sync configuration ](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eu1mkbh6vsufp2pjgun4.png)

Click "Details >" link and you will see FlightHub Sync configuration divided into sections.

![FlightHub Sync configuration details](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wzffvfx2ou7siy81ldhn.png)

We are interested in "Basic Information" and "Media File Direct Transfer" settings. Let's explore what information we will have to provide. 

### Basic Information

![Basic Information](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jglzx93rybu4v6ac6mxa.png)

We have to configure a few things
- Organization Key
- Third-Party Cloud Name
- Webhook URL

Format will be somethings like this:

```json
{
  "Organization Key": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "Third-Party Cloud Name": "My app",
  "Webhook URL": "https://xxxxxxxxxx.execute-api.eu-west-1.amazonaws.com/prod/dji/flighthub2/notify"
}
```

We can find more info about this in [Configure Information on FlightHub Sync](https://fh.dji.com/user-manual/en/flighthub-sync/introduction.html#configure-information-on-flighthub-sync).

#### Organization Key

"Organization Key" is just a very long (64 hexadecimal digits) unique ID for the organization, which is generated by FlightHub 2.

It's used for example in [FlightHub Sync APIs](https://fh.dji.com/user-manual/en/flighthub-sync/api-services.html#flighthub-sync-apis). When you call an API endpoint e.g. [Get Task Details](https://apifox.com/apidoc/shared-449a3597-cf41-4ae9-b882-00ece041c803/api-138231599) - you have to provide it as a header parameter.

![FlightHub Sync API header params](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p6dxsdlbwh5fvkcsug6e.png)

#### Third-Party Cloud Name

It's a name of the third-party cloud platform, so an arbitrary name of your application. Let's use something like "My app".

#### Webhook URL

It's a unique URL for receiving notifications from FlightHub 2, so the endpoint which we have to provision.

When media files are uploaded - FlightHub 2 will send a POST request to notify us about synced files. Body of the callback will be similar to this:

```json
{
  "notify_type": "way_line_file_upload_complete",
  "org_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "org_name": "My Organization",
  "prj_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "prj_name": "My Project",
  "sn": "XXXXXXXXXXXXXX",
  "task_info": {
    "task_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    "task_type": "way_line",
    "tags": []
  },
  "files": [
    {
      "id": 123456,
      "uuid": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "file_type": 10,
      "sub_file_type": 0,
      "name": "DJI_20240329091034_0001",
      "key": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/DJI_20240329091034_0001.jpeg"
    },
    {
      "id": 123457,
      "uuid": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "file_type": 10,
      "sub_file_type": 0,
      "name": "DJI_20240329091112_0002",
      "key": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/DJI_20240329091112_0002.jpeg"
    }
  ],
  "folder_info": {
    "expected_file_count": 2,
    "uploaded_file_count": 2,
    "folder_id": 123458
  }
}
```

## Media File Direct Transfer

![Media File Direct Transfer](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qwtjdc8xt4qv0ttj8cee.png)

![Storage Location](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j214p2rwav13u4815d9w.png)

We have to configure only one thing
- Storage Location

It's a JSON string, like this:

```json
{
  "access_key_id": "XXXXXXXXXXXXXXXXXXXX",
  "access_key_secret": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "region": "eu-west-1",
  "bucket": "flighthub2-xxxxxxxxxxxx",
  "arn": "arn:aws:iam::xxxxxxxxxxxx:role/flighthub2",
  "role_session_name": "flighthub2",
  "provider": "aws" 
}
```

We can find more info about this configuration in [Configure Media File Direct Transfer](https://fh.dji.com/user-manual/en/flighthub-sync/introduction.html#configure-media-file-direct-transfer) and [FlightHub Sync Documentation](https://sdk-forum.dji.net/hc/en-us/articles/23249981949337-FlightHub-Sync).

### Storage Location

Storage location is a configuration of OSS (Object Storage Service).

According to [FlightHub Sync Documentation](https://sdk-forum.dji.net/hc/en-us/articles/23249981949337-FlightHub-Sync) - currently supported OSS solutions are

- AWS ([Simple Storage Service](https://aws.amazon.com/s3/))
- Alibaba Cloud ([Object Storage Service](https://www.alibabacloud.com/en/product/object-storage-service))

### Project

To make it work, we will have to also enable Media File Direct Transfer on the FlightHub 2 project level.

![FlightHub 2 project configuration](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0v6v57plso44n0s59b01.png)

Important thing to note is that:

> When enabled, data collected by dock will only be uploaded to My app. DJI FlightHub 2 will not receive any data

## Infrastructure

It's now clear that we have to build a cloud based infrastructure to integrate our solution with FlightHub Sync.

Two high level building blocks will be
- OSS (Object Storage Service)
- API (Application Programming Interface)

Let's build our solution using AWS. We will use [AWS CDK (Cloud Development Kit)](https://aws.amazon.com/cdk/) and [TypeScript](https://www.typescriptlang.org/) to define our cloud application resources.

### AWS

#### OSS (Object Storage Service)

Our OSS solution will include two things: S3 bucket where media files will be uploaded and access configuration using IAM.

##### S3 (Simple Storage Service)

Let's start with defining `flighthub2-xxxxxxxxxxxx` bucket, where media files will be uploaded by Media File Direct Transfer feature of FlightHub Sync. Bucket name must be globally unique, so we will add account number as a postfix.

s3.cdk.ts
```ts
import { Construct } from "constructs";
import * as cdk from "aws-cdk-lib";
import * as s3 from "aws-cdk-lib/aws-s3";

export class S3CdkConstruct extends Construct {
  public bucket: s3.Bucket;

  constructor(scope: Construct, id: string) {
    super(scope, id);

    this.bucket = new s3.Bucket(this, "bucket", {
      bucketName: `flighthub2-${cdk.Stack.of(this).account}`,
    });

    new cdk.CfnOutput(this, "bucket", {
      value: this.bucket.bucketName,
    });
  }
}
```

##### IAM (Identity and Access Management)

Now let's define `flighthub2` user and role. User will not have access to anything, but using [AWS STS (Security Token Service)](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html) he will be able to assume `flighthub2` role and get temporary security credentials with access to the [Amazon S3](https://aws.amazon.com/s3/) bucket.

iam.cdk.ts
```ts
import { Construct } from "constructs";
import * as cdk from "aws-cdk-lib";
import * as iam from "aws-cdk-lib/aws-iam";

export class IamCdkConstruct extends Construct {
  constructor(scope: Construct, id: string) {
    super(scope, id);

    const user = new iam.User(this, "User", {
      userName: "flighthub2",
    });

    const accessKey = new iam.CfnAccessKey(this, "CfnAccessKey", {
      userName: user.userName,
    });

    new cdk.CfnOutput(this, "access_key_id", { value: accessKey.ref });
    new cdk.CfnOutput(this, "access_key_secret", {
      value: accessKey.attrSecretAccessKey,
    });

    const role = new iam.Role(this, "role", {
      assumedBy: new iam.ArnPrincipal(user.userArn),
      roleName: "flighthub2",
      managedPolicies: [
        // TODO: restrict it to flighthub2-xxxxxxxxxxxx bucket and s3:PutObject action
        iam.ManagedPolicy.fromAwsManagedPolicyName("AmazonS3FullAccess")
      ],
    });

    new cdk.CfnOutput(this, "arn", { value: role.roleArn });
  }
}

``` 

Keep in mind that we attached `AmazonS3FullAccess` policy to the `flighthub2` role. For the production ready solution we should limit access to the `flighthub2` bucket we created and allow only uploading files, so `s3:PutObject` action. This is example of "Principle of least privilege".

#### API (Application Programming Interface)

API will consist of a single endpoint, which will get notification about uploaded files. We will build it using API Gateway and a Lambda function.

##### API Gateway

Let's start with defining REST API using [API Gateway](https://aws.amazon.com/api-gateway/)

gw.cdk.ts

```ts
import * as cdk from "aws-cdk-lib";
import * as gw from "aws-cdk-lib/aws-apigateway";
import { Construct } from "constructs";

export class ApiGatewayCdkConstruct extends Construct {
  public api: gw.RestApi;

  constructor(scope: Construct, id: string) {
    super(scope, id);

    this.api = new gw.RestApi(this, "my-app");
  }
}
```

Next we will create our endpoint by defining resources and method for our webhook and integrating a lambda function with it. 

api.cdk.ts

```ts
import * as gw from "aws-cdk-lib/aws-apigateway";
import * as path from "path";
import { Construct } from "constructs";

import { ApiGatewayCdkConstruct } from "../../../gw.cdk";
import { LambdaCdkConstruct } from "../../../lambda.cdk";

interface Props {
  apigateway: ApiGatewayCdkConstruct;
}

export class ApiCdkConstruct extends Construct {
  constructor(scope: Construct, id: string, { apigateway: { api } }: Props) {
    super(scope, id);

    const dji = api.root.addResource("dji", {
      // TODO: restrict CORS
      defaultCorsPreflightOptions: {
        allowHeaders: gw.Cors.DEFAULT_HEADERS,
        allowMethods: gw.Cors.ALL_METHODS,
        allowOrigins: gw.Cors.ALL_ORIGINS,
      },
    });
    const flighthub2 = dji.addResource("flighthub2");
    const notify = flighthub2.addResource("notify");

    const notifyLambda = new LambdaCdkConstruct(this, "Notify", {
      name: "dji-flighthub2-notify",
      description: "Notification from DJI FlightHub 2",
      entry: path.join(__dirname, "./functions/notify.lambda.ts"),
    });

    notify.addMethod("POST", new gw.LambdaIntegration(notifyLambda.function));
  }
}

```

Keep in mind that CORS configuration is very permissive. For the production ready solution we should restrict it.

##### Lambda

Lambda function will be a core of our workflow. It will get a notification about uploaded media files. What we will do next highly depends on our use case. We can for example process uploaded files (generate thumbnail, detect objects), notify other parts of our app about synced files e.g. using [Amazon EventBridge](https://aws.amazon.com/eventbridge/) etc.

There are also some requirements about response, which we have to send to FlightHub Sync. According to the [FlightHub Sync Documentation](https://sdk-forum.dji.net/hc/en-us/articles/23249981949337-FlightHub-Sync) we have to respond with HTTP status code `200` and return `{ code: 0 }`.

notify.lambda.ts

```ts
import { APIGatewayProxyHandler } from "aws-lambda";

import { Notification } from "../notification";

export const handler: APIGatewayProxyHandler = async (event) => {
  console.log(JSON.stringify(event));

  let notification: Notification;
  try {
    notification = JSON.parse(event.body) as Notification;
  } catch (error) {
    console.error(error);
    return {
      statusCode: 400,
      body: JSON.stringify({ code: 1, message: "JSON parse error" }),
    };
  }

  switch (notification.notify_type) {
    case "way_line_file_upload_complete":
      // TODO: your workflow
      break;
    default:
      console.log("Unknown notification type", notification.notify_type);
      break;
  }

  return {
    statusCode: 200,
    body: JSON.stringify({ code: 0 }),
  };
};

```

notification.ts

```ts
export interface Notification {
  notify_type: "way_line_file_upload_complete" | string;
  org_id: string;
  org_name: string;
  prj_id: string;
  prj_name: string;
  sn: string;
  task_info: TaskInfo;
  files: File[];
  folder_info: FolderInfo;
}

export interface TaskInfo {
  task_id: string;
  task_type: "way_line" | string;
  tags: string[];
}

export interface File {
  id: number;
  uuid: string;
  file_type: number;
  sub_file_type: number;
  name: string;
  key: string;
}

export interface FolderInfo {
  expected_file_count: number;
  uploaded_file_count: number;
  folder_id: number;
}
```

### Testing

After deploying our application to the AWS cloud. It's time for some testing.

Before we do a final end-to-end test with a real drone and FlightHub 2, we can write an integration test, which will mimic what FlightHub Sync is doing.

### Integration Test

Let's create a simple Node.js app which will upload a file to our S3 bucket and call our webhook. 

```ts
async function mediaFileDirectTransfer(): Promise<void> {
  try {
    const credentials = await getCredentialsFromAssumedRole();
    await uploadToS3(credentials);
    await sendNotification();
  } catch (error) {
    logger.error(error);
  }
}
```

We are doing three things here
- getting credentials from the assumed role using STS
- uploading test file to S3 using those credentials
- sending notification to our API endpoint

When we run the app, we should see 3 things:
- file should be uploaded to our S3 bucket
- call to our API endpoint should return HTTP status code `200` and return `{ code: 0 }`
- we should see logs with our notification in [Amazon CloudWatch](https://aws.amazon.com/cloudwatch/)

Let's see in details how the above functions are implemented.

#### Get credentials

```ts
async function getCredentialsFromAssumedRole(): Promise<Credentials> {
  const sts = new STSClient({
    region: process.env.REGION,
    credentials: {
      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
    },
  });

  const { Credentials } = await sts.send(
    new AssumeRoleCommand({
      RoleArn: process.env.ARN,
      RoleSessionName: process.env.ROLE_SESSION_NAME
    })
  );
  logger.info(Credentials);

  return Credentials;
}
```

#### Upload file to S3

```ts
async function uploadToS3(credentials: Credentials): Promise<void> {
  const s3 = new S3Client({
    region: process.env.REGION,
    credentials: {
      accessKeyId: credentials.AccessKeyId,
      secretAccessKey: credentials.SecretAccessKey,
      sessionToken: credentials.SessionToken,
    },
  });

  const response = await s3.send(
    new PutObjectCommand({
      Bucket: process.env.BUCKET,
      Key: "fh2.txt",
      Body: `Hello from FlighHub 2 ${new Date().toISOString()}`,
    })
  );

  logger.info(response);
}
```

#### Send notification

```ts
async function sendNotification(): Promise<void> {
  const response = await axios.post(process.env.API_URL, notification);
  logger.info(response.data);
}
```

`notification` is just a JSON file like this:

notification.json
```json
{
  "notify_type": "way_line_file_upload_complete",
  "org_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "org_name": "My Organization",
  "prj_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "prj_name": "My Project",
  "sn": "XXXXXXXXXXXXXX",
  "task_info": {
    "task_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    "task_type": "way_line",
    "tags": []
  },
  "files": [
    {
      "id": 123456,
      "uuid": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "file_type": 10,
      "sub_file_type": 0,
      "name": "fh2",
      "key": "fh2.txt"
    }
  ],
  "folder_info": {
    "expected_file_count": 1,
    "uploaded_file_count": 1,
    "folder_id": 123456
  }
}
```

#### Environment variables

Our `.env` file (with environment variables) will have a structure very similar to the FlightHub Sync configuration. We can take values from the AWS CDK/CloudFormation outputs after we deployment our solution using `cdk deploy`.

.env
```
AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXX
AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
REGION=eu-west-1
BUCKET=flighthub2-xxxxxxxxxxxx
ARN=arn:aws:iam::xxxxxxxxxxxx:role/flighthub2
ROLE_SESSION_NAME=flighthub2
PROVIDER=aws
API_URL=https://xxxxxxxxxx.execute-api.eu-west-1.amazonaws.com/prod/dji/flighthub2/notify
```

## Conclusion

Although FlightHub Sync is still in beta, it has some bugs and I definitely have a wishlist of improvements (I sent feedback to the DJI) - it works and opens a lot of new possibilities for a custom drone related workflows.