Visual Guides For Any Skill With Cloudflare AI
Tags: cloudflarechallenge, devchallenge, ai
Published: 2024-04-13 12:37:57

*This is a submission for the [Cloudflare AI Challenge](https://dev.to/devteam/join-us-for-the-cloudflare-ai-challenge-3000-in-prizes-5f99).*

## What I Built

<!-- Tell us what your app does! -->
My app, Guide-ify, makes a visual guide for a given life. This is done with a 3-step process:
1. Make a request to research more about the given skill.
2. Summarize these findings.
3. Create a corresponding visual aid for each step in the summary.

These steps each correspond to a different Cloudflare AI model (see "Using AI Models" below for more details).

## Demo

<!-- Share a link to your deployed solution on Workers or Pages, and add a screenshot or video embed of your app here. -->

You can try out the app here: https://cloudflare-playground-bn1.pages.dev/

![App demo](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rkl9rw92v0tw2evoiipa.png)

## My Code

<!-- Show us the code! Share a public link to your repo and be sure to include a README file with installation instructions. We also encourage you to add a license for your code.  --> 
Here's the repo for my project: https://github.com/MrAlexLau/cloudflare-playground

Tech stack:
- The front end app was built with Sveltekit
- Styling is done with Tailwind with some snippets used from Flowbite
- The app itself is hosted on Cloudflare Pages
- The logic to generate output is done with Cloudflare Workers

## Journey

<!-- Tell us about your process, the task types and models you used, what you learned, anything you are particularly proud of, what you hope to do next, etc. -->

My goal was to build something fun and useful that stitched together multiple AI models. One of the exciting aspects with new AI tools is how easily they can be integrated together.

#### Getting Started With Cloudflare

This was my first time using the Cloudflare's Pages and Workers, and I was pleased that they were both a breeze to use! In particular, [the guide for deploying a sveltekit page](https://developers.cloudflare.com/pages/framework-guides/deploy-a-svelte-site/) made it very quick to get a site up and running on Pages.

My next step was to submit a request to a Cloudflare Worker. The two main issues I ran into were that my requests were coming across CORS errors and that I wasn't sure how to parse the params within the worker itself.

Fortunately there was sample code for both of these issues as well. I specifically referenced the [CORS Header Proxy](https://developers.cloudflare.com/workers/examples/cors-header-proxy/) and the [Fetch JSON](https://developers.cloudflare.com/workers/examples/fetch-json/) when writing the code for my Workers. For reference, you can view all of Cloudflare's Worker examples [here](https://developers.cloudflare.com/workers/examples/) as well as longer form [tutorials](https://developers.cloudflare.com/workers/tutorials/).

#### Using AI Models

My app uses 3 task types of [Cloudflare's AI models](https://developers.cloudflare.com/workers-ai/models):
1. Text-to-Text (for researching the given topic)
2. Summarization (for summarizing the research results if the given research can't be easily parsed)
3. Text-to-Image (for creating images for each step in the guide)

The code for these workers ended up being quite straightforward. For example - within a Worker, calling the Text-to-Text model is as simple as:

```javascript
let response = await ai.run('@cf/mistral/mistral-7b-instruct-v0.1', { prompt: "Tell me everything there is to know about turtles." });
```

I decided to create separate Workers for each task, which results in a separate REST endpoint that I can call for each of these operations. This worked well since I was able to test them independently and makes them more composable going forward (eg - I could create another app that uses the same Workers without having to change them at all!).

One thing that's really nice about working with Cloudflare Workers is that the editor has the ability to test out the Worker on the fly before you deploy it. Specifically you can log output (as shown in the red boxes) and preview output (as shown in the blue boxes):

![Cloudflare Worker editor](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9mfruk8b065ynovu17yn.png)


#### Handling Inconsistent Output

If you work heavily with any text-based AI model, one issue you're bound to run into is output data inconsistency. Specifically, my prompt to the models would request something like "Give me an list of actionable steps for learning how to juggle." Sometimes the output would be a numeric list, other times it would be a bulleted list or simply be a paragraph of text. The AI model being called also affected the output, for example the `@cf/tiiuae/falcon-7b-instruct` model seems to prefer bulleted lists.

I needed some way of ensuring that the data I parsed could be read reliably every time while still making sure I was preserving the intended content of the message. My approach to normalizing this data was with a 2-step process:

1. Check and see if the output text from the Text-to-Text model is a numbered list. If it is, parse out each step in that list.
2. If the output is _not_ a numbered list, then send a request to the [Summarization AI model](https://developers.cloudflare.com/workers-ai/models#summarization).

Using the Summarization model is especially useful since strips out any exclamations added in by the Text-to-Text model. For example, I noticed that the llama text model will often add phrases like `*giggles*` to its output which aren't relevant for my purposes. Using the summarization model strips out this type of extraneous text as well as preserving the main points from the text model's output.

#### Next Steps
Being that this is a project that's submitted for a hackathon, it's not production-ready code. My next steps for cleanup would be to:
- Restrict CORS settings used by my Worker endpoints. Right now they're open to make development easier, but that's not a practice I'd ever use for a production-grade application.
- Add error handling and tests.
- Moving worker logic directly into my project and developing via [Cloudflare's Wrangler](https://developers.cloudflare.com/workers/wrangler/).
  - I developed my workers directly within the Cloudflare web ui which worked well for my use case, but I feel like Wrangler would be a more smooth developer experience going forward since I wouldn't have to leave my text editor when modifying Worker code.
  - You can find the source code for my Workers [here](https://github.com/MrAlexLau/cloudflare-playground/blob/main/workers.md).

#### What I Learned
There was quite a bit that I learned with this project:
- Working within Cloudflare
  - How to deploy to Pages and Workers
  - Using the Workers web editor to modify, preview, and deploy updates
  - I like that the code for Workers is version controlled on the Cloudflare side, you can easily roll back to a previous version if you'd like
  - I was surprised that the free tier comes with **100k Worker requests per day** for on the free plan
  - Having set up these tools now, I feel like creating a new project with Pages and Workers would be _very_ fast
- Sharpening up my Svelte and Tailwind skills
- Submitting requests and reading responses from Cloudflare AI Models. For example, here's how to turn binary image output into a url that my app can use:
```javascript
const imageSrc = await fetch(
  // This worker calls the Text-to-Image `@cf/stabilityai/stable-diffusion-xl-base-1.0` AI model
  "https://text-to-image.mralexlau.workers.dev",
  requestOptions({ prompt: sentence })
).then((response) => {
    // response is binary output
    return response.blob();
}).then(function(blob) {
  const objectURL = URL.createObjectURL(blob);

  // now the return value will be a string that an <img> tag can use for its src attr
  return objectURL;
});
```

**Multiple Models and/or Triple Task Types**
Again my app uses 3 task types of [Cloudflare's AI models](https://developers.cloudflare.com/workers-ai/models). Specifically the models that I used were:

- **Summarization** - `@cf/facebook/bart-large-cnn`
- **Text-to-Image** - `@cf/stabilityai/stable-diffusion-xl-base-1.0`
- **Text-to-Text** - Defaults to `@cf/tiiuae/falcon-7b-instruct`

I also noticed that many of the text models have similar interfaces, so I added a setting to try out other text models. Users can select any of these text models:

```
@cf/meta/llama-2-7b-chat-fp16
@cf/meta/llama-2-7b-chat-int8
@cf/mistral/mistral-7b-instruct-v0.1
@cf/tiiuae/falcon-7b-instruct
@hf/google/gemma-7b-it
@hf/nousresearch/hermes-2-pro-mistral-7b
```

![Selecting a text model](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/41ixvyncbhit757inink.png)

## Wrapping Up
I had fun on this project! Using a new ecosystem can be daunting at times, but I felt like I was able to get up and running quickly and that future projects would be a breeze to set up. Looking forward to seeing what other AI models Cloudflare adds in the future.

If you have any questions about the code or running this project, please reach out in the comments!

<!-- Let us know if your project utilized multiple models per task and/or if your project used three or more task types. If so, you may qualify for our additional prize categories! If not, please remove this section. -->

<!-- Team Submissions: Please pick one member to publish the submission and credit teammates by listing their DEV usernames directly in the body of the post. -->

<!-- Don't forget to add a cover image or an additional tag to this post (if you want). -->

<!-- Thanks for participating! -->