Open AI with Vercel: Solution to Gateway Timeouts
Tags: vercel, ai, programming, webdev
Published: 2024-05-31 00:24:28

**Vercel and OpenAI: Not a Perfect Match.** 

Vercel is great for starting small projects quickly and for free. However, using AI, like OpenAI, with Vercel can cause issues that only show up when you deploy.

![Works on my machine meme](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/m9m8ek51yv7iqgqrg5hp.png)

Fixing these errors can take up a lot of our precioussss time. Here’s a quick checklist before we start:

* Are enviromental variables set?
* Is database access allowed?
* Does CORS allow origin?

## The Solution

To avoid Vercel's Gateway Timeout problems, split the OpenAI API call into two endpoints and one async function:

* Endpoint `/request/initiate`
* Function `processRequestInBackground`
* Endpoint `/request/status/:hash`

## Starting a Request

The endpoint `/request/initiate` begins a new request. It takes a user's message, creates a unique hash, and checks the database for an existing request. If found, it informs the user of the status. If new, it saves it as 'pending' and starts processing in the background.


```javascript
app.post('/request/initiate', async (req, res) => {
    const { message } = req.body;
    const hash = crypto.createHash('sha256').update(message).digest('hex');

    try {
        const client = await getDbClient();
        const db = client.db('my_db_name');
        const requestsCollection = db.collection('requests');
        const resourcesCollection = db.collection('resources');

        const existingRequest = await requestsCollection.findOne({ hash });
        if (existingRequest) {
            return res.json({ hash, status: existingRequest.status });
        }

        await requestsCollection.insertOne({ hash, message, status: 'pending' });
        res.status(202).json({ hash, status: 'pending' });

        processRequestInBackground(message, hash, requestsCollection, resourcesCollection);

    } catch (error) {
        console.error('DB error on request initiation:', error);
        res.status(500).json({ message: 'Database error during request initiation.' });
    }
});
```

## Background Processing

The `processRequestInBackground` function handles request processing. It sends the message to the OpenAI API, then updates the database with the result. If successful, it records the content and tokens used. On error, it updates the status and logs the error.

``` javascript
async function processRequestInBackground(message, hash, requestsCollection, resourcesCollection) {
    try {
        const headers = {
            "Content-Type": "application/json",
            "Authorization": `Bearer YOUR_OPEN_AI_API_KEY`
        };

        const payload = {
            model: 'gpt-4o',
            messages: [
                { role: 'system', content: YOUR_GPT_INSTRUCTIONS },
                { role: 'user', content: message }
            ],
            max_tokens: 1000
        };

        const response = await fetch('https://api.openai.com/v1/chat/completions', {
            method: 'POST',
            headers,
            body: JSON.stringify(payload)
        });

        const jsonResponse = await response.json();
        if (!response.ok) {
            throw new Error('API request failed: ' + jsonResponse.error.message);
        }

        const tokens = jsonResponse.usage.total_tokens;
        const content = jsonResponse.choices[0].message.content;
        
        await resourcesCollection.updateOne({ resource: 'openai' }, { $inc: { tokens } });
        await requestsCollection.updateOne({ hash }, { $set: { status: 'completed', content, tokens } });

    } catch (error) {
        console.error('Error during request processing:', error);
        await requestsCollection.updateOne({ hash }, { $set: { status: 'error', errorDetails: error.message } });
    }
}
```

## Checking Request Status

The endpoint `/request/status/:hash` checks the status of a specific request. It returns the current status, content, and tokens used. If no record is found, it informs the user.

``` javascript
app.get('/request/status/:hash', async (req, res) => {
    const hash = req.params.hash;

    try {
        const client = await getDbClient();
        const db = client.db('my_db_name');
        const requestsCollection = db.collection('requests');
        const requestRecord = await requestsCollection.findOne({ hash });

        if (!requestRecord) {
            return res.status(404).json({ message: 'Request record not found.' });
        }

        res.json({ status: requestRecord.status, content: requestRecord.content || null, tokens: requestRecord.tokens || 0 });

    } catch (error) {
        console.error('Error retrieving request status:', error);
        res.status(500).json({ message: 'Error processing your request', details: error.message });
    }
});
```

## And That's It!

If you enjoy JavaScript and want to join our open-source community at Webcrumbs, check out our [Discord](https://discord.com/invite/ZCj5hFv8xV), [GitHub](https://github.com/webcrumbs-community/webcrumbs), and [Waitlist](https://www.webcrumbs.org/). Follow us for more posts like this one.

See you all soon!