Repository: ollama
Issue: Llama3.1 seems broken
Labels: bug

### What is the issue?

Running on MacStudio m1, fresh pull with `ollama run llama3.1`.
Using next as a test:
```
I have 10 apples. I find 3 gold coins in the bottom of a river. The river runs near a big city that has something to do with what I can spend the coins on. I then lose 4 apples but gain a gold coin. Three birds run into my path and drop 6 apples each. I play an online game and win 6 gold coins but I have to share them equally with my 2 teammates. I buy apples for all the coins I have. The price of an apple is 0.5 coins. How many apples do I have? And where is the river?
```
Expected results are 36/unknown location for river.
Actual results:
```
ollama run llama3.1
pulling manifest
pulling 87048bcd5521... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 4.7 GB
pulling 8cf247399e57... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ 1.7 KB
pulling f1cd752815fc... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏  12 KB
pulling 56bb8bd477a5... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏   96 B
pulling e711233e7343... 100% ▕███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏  485 B
verifying sha256 digest
writing manifest
removing any unused layers
success
>>> I have 10 apples. I find 3 gold coins in the bottom of a river. The river runs near a big city that has something to do with what I can spend the coins on. I then lose 4 apples but gain a gold coin. Three birds run into my path and drop 6 apples each.
...  I play an online game and win 6 gold coins but I have to share them equally with my 2 teammates. I buy apples for all the coins I have. The price of an apple is 0.5 coins. How many apples do I have? And where is the river?
You started with 10 apples, found 3 gold coins and lost 4 apples, gained 1 coin, collected a bunch of apples from birds and bought more apples with your coins.

First, let's calculate how many apples you got from the birds. Three birds dropped 6 apples each, so that's 18 apples.
So now we have:10 (initial apples) - 4 (lost apples) + 1 (gained apple) + 18(apples from birds)=25 apples.

Next, let's calculate how many gold coins you have in total. You found 3, gained 1 and won 6 but had to share them with your teammates so you got 6/3=2.
So now we have:3 (initial coins) + 1(gained coin)+ 2(shared coins)=6.

You spent all the coins on apples. The price of an apple is 0.5 coins, so how many apples can you buy with your gold coins?
The answer is 6/0.5=12.
But remember, you already had 25 apples and you bought more apples for your gold coins (not your apples). So the total number of apples you have now is 25+12=37.
Now about the river: It runs near a big city that has something to do with what you can spend your coins on.

>>> I have 10 apples. I find 3 gold coins in the bottom of a river. The river runs near a big city that has something to do with what I can spend the coins on. I then lose 4 apples but gain a gold coin. Three birds run into my path and drop 6 apples each.
...  I play an online game and win 6 gold coins but I have to share them equally with my 2 teammates. I buy apples for all the coins I have. The price of an apple is 0.5 coins. How many apples do I have? And where is the river?
Let's break it down step by step:

**Apples:**

* You start with 10 apples.
* You lose 4 apples, so you're left with 10 - 4 = 6 apples.
* Then, three birds drop 6 apples each, which is a total of 3 x 6 = 18 apples. So, now you have 6 + 18 = 24 apples.
* And then, you gain more apples from buying them with your coins (we'll get to that in a minute). But let's not forget about the initial 10 apples you had, which means you already had 10 apples before all this happened.

**Coins:**

* You find 3 gold coins in the river.
* You gain another coin later on, so now you have 4 coins total (3 + 1).
* Then, you win 6 more gold coins playing an online game, but you share them with your 2 teammates. That means you get to keep only half of those coins: 6 / 2 = 3.
* So now you add these new coins to the ones you already had, and you have a total of 4 (initial coins) + 3 (new coins) = 7 coins.

**Buying apples with coins:**

* You buy apples for all your coins. The price of an apple is 0.5 coins.
* Since you have 7 coins, that's how many apples you can buy: 7 / 0.5 = 14 apples.
* But remember, earlier I mentioned that you already had some apples before buying more? That means the total number of apples you now have includes both these new apples and the ones you previously accumulated.

**Final answer:**

You start with 10 apples initially. Then, you accumulate more apples from birds (24 apples) and gain more apples by buying them with your coins (14 apples). So, in total, you now have:

10 (initial apples) + 24 (apples from birds) + 14 (new apples bought) = 48 apples

**The river:**

It runs near a big city that has something to do with what you can spend the coins on... You used your gold coins to buy apples! The price of an apple was related to how many coins you had, so it was probably somewhere where you could easily trade
your coins for goods.

>>> I have 10 apples. I find 3 gold coins in the bottom of a river. The river runs near a big city that has something to do with what I can spend the coins on. I then lose 4 apples but gain a gold coin. Three birds run into my path and drop 6 apples each.
...  I play an online game and win 6 gold coins but I have to share them equally with my 2 teammates. I buy apples for all the coins I have. The price of an apple is 0.5 coins. How many apples do I have? And where is the river?
**Apples:**

* You start with 10 apples.
* You lose 4 apples, so you're left with 10 - 4 = 6 apples.
* Then, three birds drop 6 apples each, which is a total of 3 x 6 = 18 apples. So, now you have 6 + 18 = 24 apples.

**Coins:**

* You find 3 gold coins in the river.
* You gain another coin later on, so now you have 4 coins total (3 + 1).
* Then, you win 6 more gold coins playing an online game, but you share them with your 2 teammates. That means you get to keep only half of those coins: 6 / 2 = 3.
* So now you add these new coins to the ones you already had, and you have a total of 4 (initial coins) + 3 (new coins) = 7 coins.

**Buying apples with coins:**

* You buy apples for all your coins. The price of an apple is 0.5 coins.
* Since you have 7 coins, that's how many apples you can buy: 7 / 0.5 = 14 apples.
* But remember, you already had some apples before buying more? That means the total number of apples you now have includes both these new apples and the ones you previously accumulated.

**Final answer:**

You start with 10 apples initially. Then, you accumulate more apples from birds (24 apples) and gain more apples by buying them with your coins (14 apples). So, in total, you now have:

24 (apples from birds) + 14 (new apples bought) = 38 apples

**The river:**

It runs near a big city that has something to do with what you can spend the coins on...
```
So no correct calculations after first three runs.
Feeding same model to llama.cpp (llama-cli --version
version: 3497 (ed9d2854)
built with Apple clang version 15.0.0 (clang-1500.3.9.4) for arm64-apple-darwin23.4.0):
```
llama-cli -m /Users/sergeymoiseev/.ollama/models/blobs/sha256-87048bcd55216712ef14c11c2c303728463207b165bf18440b9b84b07ec00f87 -c 8192 -ngl 33 -cnv
...
== Running in interactive mode. ==
 - Press Ctrl+C to interject at any time.
 - Press Return to return control to the AI.
 - To return control without starting a new line, end your input with '/'.
 - If you want to submit another line, end your input with '\'.


> I have 10 apples. I find 3 gold coins in the bottom of a river. The river runs near a big city that has something to do with what I can spend the coins on. I then lose 4 apples but gain a gold coin. Three birds run into my path and drop 6 apples each. I play an online game and win 6 gold coins but I have to share them equally with my 2 teammates. I buy apples for all the coins I have. The price of an apple is 0.5 coins. How many apples do I have? And where is the river?
You started with 10 apples and lost 4 of them. So now you have 10 - 4 = 6 apples.
You gained 1 coin. You had 3 coins before that. So now you have 3 + 1 = 4 coins.
Three birds gave you a lot of apples. Since each bird dropped 6 apples, you got 3 x 6 = 18 apples. So now you have 6 + 18 = 24 apples.
You had 4 coins and you won 6 coins. You shared the 6 coins equally with your 2 teammates. So that's 6 / 3 = 2 coins you got to keep. Now you have 4 + 2 = 6 coins.
You have 24 apples and 6 coins. The price of one apple is 0.5 coins. So the number of apples you can buy is 6 / 0.5 = 12.
So now you have 24 - 12 = 12 apples left. You had 12 apples at the end.

You can buy apples with the coins in any city. The river could be near any city. The river could be near a city called "Apple City" or "Gold Coast" or "Pigeonville." It could be anywhere.

> I have 10 apples. I find 3 gold coins in the bottom of a river. The river runs near a big city that has something to do with what I can spend the coins on. I then lose 4 apples but gain a gold coin. Three birds run into my path and drop 6 apples each. I play an online game and win 6 gold coins but I have to share them equally with my 2 teammates. I buy apples for all the coins I have. The price of an apple is 0.5 coins. How many apples do I have? And where is the river?
You started with 10 apples. You found 3 gold coins in the river, but that doesn't affect the number of apples.

You lost 4 apples, but gained 1 gold coin. So now you have 10 - 4 = 6 apples and 3 + 1 = 4 gold coins.

The three birds dropped 6 apples each, so you got 3 x 6 = 18 apples. Now you have 6 + 18 = 24 apples and still 4 gold coins.

You played the online game and won 6 gold coins. You shared them equally with your 2 teammates, so you got 6 / 3 = 2 gold coins. Now you have 4 + 2 = 6 gold coins.

You have 24 apples and 6 gold coins. The price of an apple is 0.5 coins. So, the number of apples you can buy is 6 / 0.5 = 12.

You can buy 12 apples with the 6 gold coins, so you'll have 24 - 12 = 12 apples left.

As for where the river is, the problem states that it runs near a big city that has something to do with what you can spend the coins on. It doesn't specify what the city is, but it does say that you can spend the coins on apples, and apples are related to the city of Apple City.

> I have 10 apples. I find 3 gold coins in the bottom of a river. The river runs near a big city that has something to do with what I can spend the coins on. I then lose 4 apples but gain a gold coin. Three birds run into my path and drop 6 apples each. I play an online game and win 6 gold coins but I have to share them equally with my 2 teammates. I buy apples for all the coins I have. The price of an apple is 0.5 coins. How many apples do I have? And where is the river?
You started with 10 apples.

You found 3 gold coins in the river, so you have 3 gold coins now.

You lost 4 apples, so you have 10 - 4 = 6 apples now.

You gained 1 gold coin, so now you have 3 + 1 = 4 gold coins.

You got 18 apples from the three birds, so now you have 6 + 18 = 24 apples.

You played the online game and won 6 gold coins, but you had to share them equally with your 2 teammates, so you got 6 / 3 = 2 gold coins. Now you have 4 + 2 = 6 gold coins.

You have 24 apples and 6 gold coins. The price of an apple is 0.5 coins. So, the number of apples you can buy is 6 / 0.5 = 12.

You can buy 12 apples with the 6 gold coins, so now you have 24 - 12 = 12 apples left.

The river is near a big city where you can spend the gold coins. Since you spent the gold coins on apples, it's likely that the city is Apple City or somewhere related to apples.
```

### OS

macOS

### GPU

Apple

### CPU

Apple

### Ollama version

0.3.2