ChatGPT vs Llama solving Einstein's famous riddle
Tags: ai, chatgpt, llama, testing
Published: 2023-11-17 16:54:00

There is a famous riddle that was supposed to be invented by Albert Einsten, although there is no evidence that he did, it is still a fun riddle and a very difficult one to solve!

I wanted to get an idea of the problem-solving ability from both models (Chat GPT and Llama) to check how good they are. I look forward for the ability of AI where you can submit these type of "clues" from a real world problem and it will help you to solve them using natural language.

Here is the riddle and the exact prompt I submitted to both models. By the way, I used ChatpGPT 3.5 and Llama 2.

There are five houses lined up next to each other along a street. Each house is a different color, and each homeowner is of a different nationality, drinks a different beverage, smokes a different brand of cigar, and owns a different pet.

If these variables can never repeat from house to house, which homeowner has a pet fish? You should be able to figure out the answer based on these 15 clues:

- The Englishman lives in the house with red walls.
- The Swede keeps dogs.
- The Dane drinks tea.
- The house with green walls is just to the left of the house with white walls.
- The owner of the house with green walls drinks coffee.
- The man who smokes Pall Mall keeps birds.
- The owner of the house with yellow walls smokes Dunhills.
- The man in the center house drinks milk.
- The Norwegian lives in the first house.
- The Blend smoker has a neighbor who keeps cats.
- The man who smokes Blue Masters drinks beer.
- The man who keeps horses lives next to the Dunhill smoker.
- The German smokes Prince.
- The Norwegian lives next to the house with blue walls.
- The Blend smoker has a neighbor who drinks water.

Trust me, you will need a paper and a pencil to draw the houses or make a digram to solve it. Here is the response from both models.

**ChatGPT 3.5**

```
This riddle involves using logic to deduce the attributes of each house and homeowner. Let's break down the clues:

The Norwegian lives in the first house.
The house with green walls is just to the left of the house with white walls.
The owner of the house with green walls drinks coffee.
The man in the center house drinks milk.
The Norwegian lives next to the house with blue walls.
Now let's organize the information we know so far:

House 1: Norwegian, yellow walls, ???, ???, ???
House 2: ???, ???, ???, ???, ???
House 3: ???, green walls, ???, ???, ???
House 4: ???, white walls, ???, ???, ???
House 5: ???, blue walls, ???, ???, ???

From clue 1, the Norwegian lives in the first house, and from clue 5, the Norwegian lives next to the house with blue walls. This means the first house is blue.

House 1: Norwegian, blue walls, ???, ???, ???
House 2: ???, ???, ???, ???, ???
House 3: ???, green walls, ???, ???, ???
House 4: ???, white walls, ???, ???, ???
House 5: ???, blue walls, ???, ???, ???

From clue 2, the house with green walls is just to the left of the house with white walls. Since the house with blue walls is already in the first position, the house with green walls must be in the second position, and the house with white walls must be in the third position.

House 1: Norwegian, blue walls, ???, ???, ???
House 2: ???, green walls, ???, ???, ???
House 3: ???, white walls, ???, ???, ???
House 4: ???, ???, ???, ???, ???
House 5: ???, blue walls, ???, ???, ???

From clue 3, the owner of the house with green walls drinks coffee. This means the person in the second house drinks coffee.

House 1: Norwegian, blue walls, ???, ???, ???
House 2: ???, green walls, coffee, ???, ???
House 3: ???, white walls, ???, ???, ???
House 4: ???, ???, ???, ???, ???
House 5: ???, blue walls, ???, ???, ???

From clue 4, the man in the center house drinks milk. Since there are an odd number of houses, the third house must be the center one, so the person in the third house drinks milk.

House 1: Norwegian, blue walls, ???, ???, ???
House 2: ???, green walls, coffee, ???, ???
House 3: Milk, white walls, ???, ???, ???
House 4: ???, ???, ???, ???, ???
House 5: ???, blue walls, ???, ???, ???

From clue 5, the Norwegian lives next to the house with blue walls, meaning the Norwegian is in the first house and the blue walls are in the second house.

House 1: Norwegian, blue walls, ???, ???, ???
House 2: ???, blue walls, ???, ???, ???
House 3: Milk, white walls, ???, ???, ???
House 4: ???, ???, ???, ???, ???
House 5: ???, blue walls, ???, ???, ???

Now, we can figure out the rest of the clues based on these deductions. The Norwegian has the blue house, drinks water, smokes Dunhills, and owns cats. Therefore, the homeowner in the first house (the Norwegian) has a pet fish.
```

**Llama 2**
Now the turn for open source Facebook AI.

```
Sure! Here's how you can solve the riddle:
1. The Englishman lives in the house with red walls.
2. The Swede keeps dogs.
3. The Dane drinks tea.
4. The house with green walls is just to the left of the house with white walls.
5. The owner of the house with green walls drinks coffee.
6. The man who smokes Pall Mall keeps birds.
7. The owner of the house with yellow walls smokes Dunhills.
8. The man in the center house drinks milk.
9. The Norwegian lives in the first house.
10. The Blend smoker has a neighbor who keeps cats.
11. The man who smokes Blue Masters drinks beer.
12. The man who keeps horses lives next to the Dunhill smoker.
13. The German smokes Prince.
14. The Norwegian lives next to the house with blue walls.
15. The Blend smoker has a neighbor who drinks water.

From clue 1, we know that the Englishman lives in the house with red walls. From clue 9, we know that the Norwegian lives in the first house. Therefore, the Norwegian must live in the house with red walls, since the Englishman lives in the house with red walls.

From clue 2, we know that the Swede keeps dogs. From clue 10, we know that the Blend smoker has a neighbor who keeps cats. Therefore, the Swede must keep dogs and the Blend smoker must have a neighbor who keeps cats.

From clue 3, we know that the Dane drinks tea. From clue 5, we know that the owner of the house with green walls drinks coffee. Therefore, the Dane must drink tea and the owner of the house with green walls must drink coffee.

From clue 4, we know that the house with green walls is just to the left of the house with white walls. From clue 14, we know that the Norwegian lives next to the house with blue walls. Therefore, the house with green walls must be to the left of the house with white walls, and the Norwegian must live next to the house with blue walls.

From clue 6, we know that the man who smokes Pall Mall keeps birds. From clue 11, we know that the man who smokes Blue Masters drinks beer. Therefore, the man who smokes Pall Mall must keep birds, and the man who smokes Blue Masters must drink beer.

From clue 7, we know that the owner of the house with yellow walls smokes Dunhills. From clue 12, we know that the German smokes Prince. Therefore, the owner of the house with yellow walls must smoke Dunhills, and the German must smoke Prince.

From clue 8, we know that the man in the center house drinks milk. 

From clue 13, we know that the man who keeps horses lives next to the Dunhill smoker. Therefore, the man in the center house must drink milk, and the man who keeps horses must live next to the Dunhill smoker.

Now, let's look at the remaining clues:

Clue 15 states that the Blend smoker has a neighbor who drinks water. This could refer to either the man who smokes Blue Masters (clue 11) or the man who keeps cats (clue 10). However, since we already know that the man who smokes Blue Masters drinks beer (clue 11), it must be the man who keeps cats who drinks water
```

Well as you can see the results are not very promising... I noticed on the first paragraph from Llama 2 that it confused the "context" of the variables, in this case the nationality. It even didn't show what the answer is.

ChatGPT in the other hand was able to separate or understand a little bit more the context of the variables. But the answer was wrong at the end.

Although I believe that we can create a prompt detailing all variables and it should help them to show the right answer, this was just a test on how a riddle is presented in the "real world" to humans and how AI will try to solve it.

We are in a very early stages with AI and we are starting to grasp its capabilities.