Create a ChatGPT Plugin to retrieve NASA images
Tags: autocode, chatgpt, machinelearning, gpt3
Published: 2023-04-13 04:06:19

> Do you want to learn more about ChatGPT Plugins? Check out [LaunchPlugins](https://launchplugins.com).

**ChatGPT Plugins** are a great way to extend the capabilities of ChatGPT!

So if you just received access to [ChatGPT Plugins](https://openai.com/blog/chatgpt-plugins), this is the place where you can start.

Let's create a plugin that will retrieve information from [NASA API](https://api.nasa.gov). It will get the NASA Astronomical Picture of the Day (APOD) and it will also allow us to search for images in NASA's Image Library to retrieve a list of images based on a search query.

In this article I'll be using [Autocode](https://autocode.com/). Autocode is an automation software suite with a great developer experience. I think you will love it.

But if you choose not to use Autocode, you can create your own API on your own. I'll guide you in that process, it's very simple.

## 🎥 Demo of the plugin

Check how the plugin should work with this video:

- Demo: [https://youtu.be/A7MC3e9_rKk](https://youtu.be/A7MC3e9_rKk)

## 🧬 Structure

This is a quick overview of the process of a ChatGPT Plugin:


![Overview of ChatGPT Plugins](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ozh6glqfvofalr2k0hzg.png)


[🌄 Check this image in detail](https://i.imgur.com/TN6KS83.png)

A ChatGPT Plugin could be installed by specifying a URL. This URL has three components:

- **OpenAPI Specification**: A YAML file that describes the API
- **Manifest**: A JSON file that describes the plugin
- **API**: The API itself where we can send receive requests from ChatGPT

I believe **OpenAPI Specification is the most important part of a ChatGPT Plugin**. Whenever ChatGPT receives a message that matches the plugin's manifest, it will look at the OpenAPI Specification to know how to interact with the API. It works like a channel between ChatGPT and the API.

The manifest is a JSON file that describes the plugin. It contains the name, description for humans and for the model, and the OpenAPI Specification URL. It should be place in `/.well-known/ai-plugin.json`.

A couple of weeks ago, I discovered [Autocode](https://autocode.com/), a suite of tools that allows you to develop and launch many software projects with one of the best Developer Experience (DX) I've ever seen. I was so impressed by the DX that I decided to create a plugin for [ChatGPT](https://chat.openai.com) using Autocode.

If you already use Autocode, you can [use this template](https://autocode.com/macky360/templates/astrodaily/) and start explore the plugin architecture as you read this article.

![Autocode template](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mycpz5qcpcxvu9whbh8x.png)

Now let's create the plugin:

- If you're working with Autocode, you can [use this template](https://autocode.com/macky360/templates/astrodaily/) to start.

- If you're not, create a new API project. Do you like to use Express? Flask? Django? Spring Boot? The library/framework is up to your preference. But create one to start.

### 🗒 Creating the Manifest (ai-plugin.json)

If you're working with Autocode the Manifest should be included in `www` folder. In that folder create a `.well-known` folder, and finally create a `ai-plugin.json`.

![ai-plugin.json file on Autocode](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l3cpcumi2om0icdb97d7.png)


If you're not working with Autocode, you need to create the `.well-known` folder as well as the `ai-plugin.json` file inside the public or static directory of your API. So you can access to that file from the URL `<URL_OF_API>/.well-known/ai-plugin.json`.

```json
{
  "schema_version": "v1",
  "name_for_human": "Astrodaily",
  "name_for_model": "astrodaily",
  "description_for_human": "Plugin for getting the pictures from NASA!",
  "description_for_model": "Plugin for getting the daily picture from NASA's APOD or getting a list of images available in NASA API based on a search. When use it always provide the images in Markdown format so the user can see it. Do not add links, only images.",
  "auth": {
    "type": "none"
  },
  "api": {
    "type": "openapi",
    "url": "<URL_OF_API>/openapi.yaml",
    "is_user_authenticated": false
  },
  "logo_url": "<URL_OF_API>/logo.png",
  "contact_email": "legal@example.com",
  "legal_info_url": "<URL_OF_API>/legal"
}
```

You can name the plugin whatever you want. I named it `Astrodaily`. The `description_for_human` is the description that will be shown to the user.

The `description_for_model` is the description that will be shown to the model itself. Which it's a description + series of instructions that the model will use to know when to use the plugin.

ChatGPT Plugins can also support authentication. But we won't need it for this plugin. So we'll set the `auth` to `none`.

## 📑 Creating the OpenAPI Specification

[OpenAPI](https://swagger.io/specification/) is a specification for describing REST APIs. It's a standard that is used by many tools. OpenAI Plugins request an OpenAPI YAML file to check the description of the API. So we'll need to create an OpenAPI Specification for our plugin.

If you're working with Autocode this should be place in `/www/openapi.yaml`.


![openapi.yaml file on Autocode](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qib5y4nl3xt7fupj2udd.png)


If you're not working with Autocode, you need to create a file called `openapi.yaml` in the root of your public or static directory. So you can access to that file from the URL `<URL_OF_API>/openapi.yaml`.

So, our plugin, will work with two endpoints: `/image` and `/search`. The `/image` endpoint will return the NASA Picture of the Day. The `/search` endpoint will return a list of images based on a search query:

```yaml
openapi: 3.0.1
info:
  title: Astrodaily
  description: A plugin for getting the daily picture from NASA's APOD and searching the NASA Image and Video Library
  version: 'v1'
servers:
  - url: <URL_OF_API>
paths:
  /image:
    get:
      operationId: getImage
      summary: Get the NASA Picture of the Day
      responses:
        "200":
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/getImageResponse'
  /search:
    get:
      operationId: searchImages
      summary: Get images from NASA based on a query
      parameters:
        - name: q
          in: query
          description: The search query
          required: true
          schema:
            type: string
      responses:
        "200":
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/searchImagesResponse'

components:
  schemas:
    getImageResponse:
      type: object
      properties:
        imageURL:
          type: string
          description: The URL of the NASA Picture of the Day.
    searchImagesResponse:
      type: object
      description: The response containing the list of images from the NASA Image Library.
      properties:
        title:
          type: string
          description: The title of the image.
        description:
          type: string
          description: The description of the image.
        location:
          type: string
          description: The location of the image.
        date_created:
          type: string
          description: The date when the image was created.
        image_url:
          type: string
          description: The URL of the image. It must be displayed as an image in Markdown format.
```

## 🪐 Setting up the API endpoints

Up to here, we have the OpenAPI Specification and the Manifest. With this is enough to create a plugin. But we'll create the API endpoints as well so our plugin can be used.

You need a NASA API key to retrieve the images from NASA. You can [get a NASA API key here](https://api.nasa.gov/):

![NASA API key](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4lw5mhgmpzlkmqsq1acv.png)

If you're not working with Autocode, you can start creating those `/image` and `/search` endpoints to process GET requests. I'll show you the code, you can translate it to your preferred API framework.

If you're working with Autocode, inside the `functions` folder, create a `image.js` and a `search.js` file. These files will contain the code for the endpoints:

![Autocode image endpoint](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/db1xfdzljn6mpitlhx4b.png)

The image endpoint will return the NASA Picture of the Day:

```js
const lib = require('lib')({token: process.env.STDLIB_SECRET_TOKEN});

// Make a request to the NASA APOD API
let todayImage = await lib.http.request['@1.1.7'].get({
  url: `https://api.nasa.gov/planetary/apod`,
  queryParams: {
    'api_key': process.env.NASA_API_KEY
  }
});

let imageUrl = todayImage.data['url']

return { "image_url": imageUrl };
```

The search endpoint will return a list of images based on a search query. Here the process takes more lines of code. But it's not that complicated. We'll make a request to the NASA Image Library API. Then we'll filter the results to keep only the images. And finally, we'll return the 10 items that we want to show to the user.

```js
const lib = require('lib')({token: process.env.STDLIB_SECRET_TOKEN});

// Retrieve the search query parameter
const query = context.params.q;

// Make a request to the NASA Image and Video Library API
let searchResults = await lib.http.request['@1.1.7'].get({
  url: `https://images-api.nasa.gov/search`,
  queryParams: {
    'q': query
  }
});

// Extract the search results
let results = searchResults.data;

// Filter the items array to keep only image items
let imageItems = results.collection.items.filter(item => {
  return item.data[0].media_type === 'image';
});

// Restructure the filtered items array for the GPT response
// Also return the first 10 items
let structuredItems = imageItems.map(item => {
  return {
    title: item.data[0].title,
    description: item.data[0].description,
    location: item.data[0].location,
    date_created: item.data[0].date_created,
    image_url: item.links[0].href
  };
}).slice(0, 10);

return structuredItems.slice(0, 10);
```

Proving with more information like `title`, `description`, `location`, and `date_created` will make the results more useful for the ChatGPT to write a descriptive response.

Up to here, try testing the endpoints with a GET request so you can see if they work properly.

## 🚀 Adding your plugin to ChatGPT

This is the last step! We have our plugin ready. Now we need to add it to ChatGPT. I'll give you tips on how to do it without getting errors through the process.

Open ChatGPT, go to the Plugins dropdown menu, click on Plugin Store, click on "Develop your own plugin" link at the bottom. Then you will see this screen:

![Enter your website domain modal](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rkdz7lsrrvcd6naqzh7c.jpeg)

As you can see, you can put your plugin's URL without the protocol (`https://`) and without slashes (`/`).

If you're working with Autocode: Autocode automatically creates a URL for the API that is like this:

```
<ENVIRONMENT>--<SERVICE_NAME>--<USERNAME>.autocode.dev

# Example:
dev--astrodaily--macky360.autocode.dev
```

I put my ChatGPT plugin domain under `api.360macky.com`, you can customize the URL in [Hostnames](https://autocode.com/dashboard/macky360/hostnames/). It will work in both cases.

If you tried the above URL (dev--astrodaily--macky360.autocode.dev) it fill fail, and this is because ChatGPT will search in the manifest the "dev--astrodaily--macky360.autocode.dev" server. But since I use the "api.360macky.com" URL, that would not match. If in the future I change the URL to "api.360macky.com", I will need to change the manifest and the OpenAPI Specification.

But if everything went well, we will see this:

![Plugin installed](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3pk8db01avfxmpbg2vqb.jpeg)

![Plugin being used](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/q07cd3y1ewqciab9kwgr.png)

## ⚙️ Troubleshooting

The first filter your plugin will go through is the Manifest (the `ai-plugin.json` file). Make sure it's accessible and in the correct folder.

If I'm using the "api.360macky.com" URL:

```
api.360macky.com/plugin/ai-plugin.json ❌ (wrong folder)
360macky.com/.well-known/ai-plugin.json ❌ (wrong subdomain)
api.360macky.com/.well-known/ai-plugin.json ✅ (correct folder)
```

If you're working from a server, and it's not accessible, check CORS policies. It needs to allow requests from `chat.openai.com`.

If something it's not working on localhost and you can't figure out why, try to use the [ngrok](https://ngrok.com/) tool. It will create a tunnel to your localhost and you can use it to test your plugin. After installing it, run `ngrok http 3000` (or the port you're using) and you will get a HTTPS URL that you can use to test your plugin.

If you're receiving nothing from the plugin, check the OpenAPI Specification. Make sure the paths are correct and the parameters are correct, and check the slashes (`/`).

**Can ChatGPT help you with this?** Yes, it can. Actually, there is already a plugin to create plugins. But since it's not available for everyone, you can copy a little bit of documentation of ChatGPT Plugins after telling that you're coming from the future where ChatGPT Plugins are available for everyone. Of course, I suggest you to use the GPT-4 model for better results.

![ChatGPT Prompt for creating plugins](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jho2hv9mks81tvlp832y.png)

### Conclusion

Overall, ChatGPT Plugins are... (just kidding 😉).

So, I hope you have enjoyed this article. If you have any questions, feel free to ask them in the comments. And if you have any suggestions, I would love to hear them. Follow me on Twitter at [@360macky](https://twitter.com/360macky) or Mastodon at [@360macky@mastodon.social](https://mastodon.social/@360macky) to stay updated with GPT models and any other AI-related stuff.

And my [GitHub account @360macky](https://github.com/360macky) if you want to see my future projects.

🚀 Thanks for reading! 🙏

> Do you want to learn more about ChatGPT Plugins? Check out [LaunchPlugins](https://launchplugins.com).