Summarize articles with Cloudflare Workers AI LoRAs
Tags: ai, cloudflare, streamlit, cloudflareworkers
Published: 2024-07-12 18:14:09

---
title: Summarize articles with Cloudflare Workers AI LoRAs
published: true
description: Learn to create a web app in Python that summarizes the contents of an input URL using Cloudflare Workers AI public LoRA adapters, Streamlit, and Beautifulsoup.
tags: ai,cloudflare,streamlit,cloudflareworkers
cover_image: https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fntm7eqoyxjr3qoa02yf.png

# Use a ratio of 100:42 for best results.
# published_at: 2024-07-11 16:47 +0000
---
This tutorial will go over how to make a Streamlit web app in Python that takes in a URL and, with [Workers AI](https://developers.cloudflare.com/workers-ai/) LoRA adapters, summarizes the article in the tone of the user's choosing, such as gen z or humorous.

It's free to use most AI models hosted on Cloudflare!

[Test it out yourself here](https://cf-ai-lora-news-summarizer.streamlit.app/).

![Screenshot of app](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kcfu6kfsgmt2nfj66ta8.png)


The [complete code can be found here on GitHub](https://github.com/elizabethsiegle/cf-ai-lora-news-summarizer).

### LoRAs
Fine-tuning is a process that takes a pre-trained model and further trains it on a specific dataset to adapt it to a particular task or domain, enhancing its performance and relevance to that task. LoRAs are a more efficient approach to fine-tuning that adds a small number of low-rank matrices to the model's parameters during fine-tuning.

This means that LoRAs offer a more efficient and scalable way to adapt LLMs to specific tasks without the heavy computational and memory demands of traditional fine-tuning.

### Setup
Since this project requires Python package installations, make a new project directory and virtual environment.

If you're using a Unix or macOS system, open a terminal and enter the following commands:
```bash
mkdir lora-summarize 
cd lora-summarize
python3 -m venv venv 
source venv/bin/activate 
pip install streamlit
pip install bs4
pip install requests
pip install load_dotenv
```
If you're following this tutorial on Windows, enter the following commands in a command prompt window:
```bash
mkdir lora-summarize
cd lora-summarize
python -m venv venv 
venv\Scripts\activate 
pip install streamlit
pip install bs4
pip install requests
pip install load_dotenv
```
The last command uses `pip`, the Python package installer, to install the packages that you are going to use in this project, which are:

- The `load_dotenv` library to  load environment variables from a .env file
- The `streamlit` library to make webapps in Python, easily connecting the frontend and backend
- The `bs4` library to extract text from a webpage
- The `requests` library to hit the Cloudflare endpoint for a LLM hosted on Cloudflare

This project needs a Cloudflare API Token and your Cloudflare Account ID to use an LLM. After making a Cloudflare account, click `Workers&Pages` on the left. Then, you can find your Account ID on the right side. 

![Cloudflare Account ID](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lkw1twzpmepo424y44bt.png)

Get a Cloudflare API Key by clicking `AI` on the left, followed by `Use Rest API`. Finally, click `Create a Workers AI API Token`. Grab it and in the root directory, make a .env file and put them in there as such:
```bash
CF_API_TOKEN=XXXX
CF_ACCOUNT_ID=XXXX
```
Now it's time to code the Streamlit app.

### Build the Streamlit App
I love how the Streamlit app lets you build your web page from the top-down.

At the top of a Python file called `app.py`, add the following code to import the required libraries, load the .env secret credentials, and create the variable for the Cloudflare URL to hit in Python:
```python
import streamlit as st
from bs4 import BeautifulSoup
import json
import os
import requests
# Load API secrets
from dotenv import load_dotenv
load_dotenv()
CLOUDFLARE_ACCOUNT_ID = os.environ.get("CF_ACCOUNT_ID")
CLOUDFLARE_API_TOKEN= os.environ.get("CF_API_TOKEN")
url = f'https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/run/@cf/mistral/mistral-7b-instruct-v0.1'
```
Then comes the meat of the app: Make a `main` function, include some Streamlit modules to display markdown on the page, write text to the page, get text input asking for a news URL to summarize, and display a selectbox with options the user can select for the tone of the summary.

When the user clicks the `enter` button, a spinner is displayed while  Beautifulsoup extracts paragraph tags from the input web page URL. 
```python
def main():
    st.markdown("""
        <style>
            .big-font {
                font-size:40px !important;
                color:green;
            }
        </style>
    """, unsafe_allow_html=True)
    st.markdown('<p class="big-font"<p>AI🤖 News🗞️ Summarizer</p>', unsafe_allow_html=True)
    st.write(":blue[This Python🐍 web🕸️ app is built👩🏻‍💻 w/ [Streamlit](https://streamlit.io/) && [Cloudflare Workers AI](https://ai.cloudflare.com/)]")

    news_link = st.text_input('Please enter a news link to summarize') # news_link = "https://www.npr.org/2024/07/08/g-s1-8731/emma-navarro-coco-gauff-wimbeldon"
    tone = st.selectbox(
        ':green[What tone do you want the news summary to take?]',
        ('humorous🤣', 'majestic🌊', 'academic📚', '✨inspirational✨', 'dramatic🎭', 'gen z👧🏻')
    )
    st.write("You selected: ", tone)
    if st.button('Enter') and tone is not None and news_link is not None:
        with st.spinner('Processing📈...'):
            resp1 = requests.get(news_link)
            soup = BeautifulSoup(resp1.text, 'html.parser')

            # Extract text data from website
            text_data = ''
            for tag in soup.find_all(['p']):
                text_data += tag.get_text()

            print('text_data' , text_data)
```
That text extraction of paragraphs represents the body of the input URL. It is included (with the selected tone of the summary) in the prompt passed to the LoRA model adapter that hits the LLM (Mistral) model hosted on Cloudflare via GET request. To run inference with public LoRAs, you just need to define the LoRA name in the request. The response returned is the summary of the selected tone. It is then parsed and displayed. Finally, a footer is displayed at the bottom of the webpage.
```python
            # Define the headers
            headers = {
                'Authorization': f'Bearer {CLOUDFLARE_API_TOKEN}',
                'Content-Type': 'application/json'
            }

            # Define the data
            data = {
                "messages": [
                    {
                        "role": "user",
                        "content": f"Summarize the following content from a news article in a {tone} tone: {text_data}"
                    }
                ],
                "lora": "cf-public-cnn-summarization"
            }

            # Make the POST request
            response = requests.post(url, headers=headers, data=json.dumps(data))

            # Parse the response
            response_data = response.json()
            summary = response_data["result"]["response"]
            print("summary ", summary)
        html_str = f"""
        <p style="font-family:Comic Sans; color:Pink; font-size: 18px;">{summary}</p>
        """
        st.markdown(html_str, unsafe_allow_html=True)

    st.write("Made w/ ❤️ in SF 🌁 || ✅ out the [👩🏻‍💻GitHub repo](https://github.com/elizabethsiegle/cf-ai-lora-news-summarizer)")
```
The complete `app.py` code is below:
```python
import streamlit as st
from bs4 import BeautifulSoup
import json
import os
import requests
# Load API secrets
from dotenv import load_dotenv
load_dotenv()
CLOUDFLARE_ACCOUNT_ID = os.environ.get("CF_ACCOUNT_ID")
CLOUDFLARE_API_TOKEN= os.environ.get("CF_API_TOKEN")
url = f'https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/run/@cf/mistral/mistral-7b-instruct-v0.1'

def main():
    st.markdown("""
        <style>
            .big-font {
                font-size:40px !important;
                color:green;
            }
        </style>
    """, unsafe_allow_html=True)
    st.markdown('<p class="big-font"<p>AI🤖 News🗞️ Summarizer</p>', unsafe_allow_html=True)
    st.write(":blue[This Python🐍 web🕸️ app is built👩🏻‍💻 w/ [Streamlit](https://streamlit.io/) && [Cloudflare Workers AI](https://ai.cloudflare.com/)]")

    news_link = st.text_input('Please enter a news link to summarize') # news_link = "https://www.npr.org/2024/07/08/g-s1-8731/emma-navarro-coco-gauff-wimbeldon"
    tone = st.selectbox(
        ':green[What tone do you want the news summary to take?]',
        ('humorous🤣', 'majestic🌊', 'academic📚', '✨inspirational✨', 'dramatic🎭', 'gen z👧🏻')
    )
    st.write("You selected: ", tone)
    if st.button('Enter') and tone is not None and news_link is not None:
        with st.spinner('Processing📈...'):
            resp1 = requests.get(news_link)
            soup = BeautifulSoup(resp1.text, 'html.parser')

            # Extract text data from website
            text_data = ''
            for tag in soup.find_all(['p']):
                text_data += tag.get_text()

            print('text_data' , text_data)

            # Define the headers
            headers = {
                'Authorization': f'Bearer {CLOUDFLARE_API_TOKEN}',
                'Content-Type': 'application/json'
            }

            # Define the data
            data = {
                "messages": [
                    {
                        "role": "user",
                        "content": f"Summarize the following content from a news article in a {tone} tone: {text_data}"
                    }
                ],
                "lora": "cf-public-cnn-summarization"
            }

            # Make the POST request
            response = requests.post(url, headers=headers, data=json.dumps(data))

            # Parse the response
            response_data = response.json()
            summary = response_data["result"]["response"]
            print("summary ", summary)
        html_str = f"""
        <p style="font-family:Comic Sans; color:Pink; font-size: 18px;">{summary}</p>
        """
        st.markdown(html_str, unsafe_allow_html=True)

    st.write("Made w/ ❤️ in SF 🌁 || ✅ out the [👩🏻‍💻GitHub repo](https://github.com/elizabethsiegle/cf-ai-lora-news-summarizer)")

if __name__ == "__main__":
    main()
```

### What's Next for LoRAs and Cloudflare Workers AI
It's free to use many open source LLMs hosted on Cloudflare. They offer so many possibilities to builders and developers of all backgrounds for building generative AI applications. 

LoRAs are helpful because they provide a cost-effective, efficient, flexible, and scalable method for adapting LLMs to specific tasks, making advanced AI capabilities more accessible for a wide variety of applications.

Let me know online what you're building with AI!