Bard vs ChatGPT: round 1
Tags: machinelearning, chatgpt
Published: 2023-05-11 20:22:49

Last night I ran a very quick, unrigorous, unscientific, and maybe even unfair comparison between Bard and ChatGPT.

LLMs do particularly well in those cases when we need something quick and simple in a tool we know little about. And I happened to have such a case: a simple website using htmx and fastAPI.

This is not going to be a long post, so I'm jumping directly to the two things that caught my attention:
- Bard missed the prompt, and ChatGPT did much better.
- ChatGPT asked me relevant questions to confirm understanding, Bard did not.

## Bard missed the prompt

I asked Bard to use htmx. It kind of complied, but not really: it linked to htmx but did not use it.

```html
    <h1>Function analyzer</h1>
    <input type="text" id="contract" placeholder="Enter a contract">
    <button id="analyze">Analyze</button>
    <div id="tree"></div>

    <script src="https://cdnjs.cloudflare.com/ajax/libs/htmx/1.8.0/htmx.min.js"></script>
```
Where are my htmx attributes Bard? All the logic is done in JS, which I wanted to avoid. 

{% details full body %} 

```html
<body>
    <h1>Function analyzer</h1>
    <input type="text" id="contract" placeholder="Enter a contract">
    <button id="analyze">Analyze</button>
    <div id="tree"></div>

    <script src="https://cdnjs.cloudflare.com/ajax/libs/htmx/1.8.0/htmx.min.js"></script>
    <script>
        const contractInput = document.getElementById("contract");
        const analyzeButton = document.getElementById("analyze");
        const treeDiv = document.getElementById("tree");

        analyzeButton.addEventListener("click", () => {
            // Get the contract from the input box.
            const contract = contractInput.value;

            // Make a request to the backend to get the tree view of the contract.
            fetch("/function-tree", {
                method: "GET",
                body: JSON.stringify({
                    contract: contract
                })
            })
            .then(response => response.json())
            .then(tree => {
                // Render the tree to HTML.
                const html = render_tree(tree);

                // Insert the HTML into the tree div.
                treeDiv.innerHTML = html;
            });
        });

        function render_tree(tree) {
            // Create a new ul element.
            const ul = document.createElement("ul");

            // Iterate over the tree.
            for (const key in tree) {
                // Create a new li element.
                const li = document.createElement("li");

                // Set the text of the li element to the key.
                li.textContent = key;

                // If the key is a dictionary, recursively render its children.
                if (typeof tree[key] === "object") {
                    li.appendChild(render_tree(tree[key]));
                }

                // Append the li element to the ul element.
                ul.appendChild(li);
            }

            // Return the ul element.
            return ul;
        }
    </script>
```
{% enddetails %}


How did ChatGPT do? It did a good job, even if I had to edit the code to fix some bugs. 
```html
    <h1>Function Analyzer</h1>
    <label for="contract">Contract:</label>
    <input type="text" id="contract">
    <button hx-post="/function-tree" hx-trigger="click" hx-target="#treeview" hx-swap="outerHTML">Analyze</button>
    <label for="search">Search:</label>
    <input type="text" id="search" hx-post="/function-tree" hx-trigger="keyup changed delay:500ms" hx-target="#treeview" hx-swap="outerHTML">
    <div id="treeview"></div>

```

Note the `hx-post` (which I changed to `hx-get`), `hx-target` and `hx-swap` which are some of the basic elements of htmx.



## ChatGPT asked good questions
My prompt included the following request in the last paragraph:
> Your first task is to confirm your understanding of the task, by asking any questions that might be necessary, and then providing a design document. Once the document is confirmed by me, you will write the code.

ChatGPT asked good, relevant questions:
```
To confirm my understanding of the task and provide a design document, I have a few questions and a proposed design.

Questions:

1. Are there any specific design requirements for the website (e.g., colors, fonts, layout)?
2. Should the search functionality filter the tree view in real-time as the user types in the search text box, or should it only filter after pressing a "Search" button?
3. Is it required to use any specific front-end libraries or frameworks, other than htmx?
```

Bard was kind of pedantic about it:
```
I understand the task as follows:
[...]
Do you have any questions about my understanding of the task?
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ljqexxxs883z2a8ikf5e.gif)

I'm giving round 1 to ChatGPT for following the prompt in terms of asking questions and using htmx as requested.