[Xiaomi 13 Ultra] lubo777:
Hello, this is a step by step guide how to run Artificial Intelligence models like the famous Mistral:7B or Llama2:7B and many more LLMs on your smartphone Xiaomi 13 Ultra.

All AI models are installed locally on your phone and are working offline.

This method is applicable to both rooted and non-rooted smartphones with enough RAM, not just the Xiaomi 13 Ultra.

In order to run the bigger (more smarter) LLM model with 7 Billion parameters you need 12 GB RAM for smooth operation.

On Xiaomi 13 Ultra we have 12 / 16 GB RAM and the 7B LLMs is working perfectly on my phone with only 12 GB of RAM.

I have tested the procedure bellow on my old Redmi Note 10 Pro which has only 6 GB RAM and the 7B parameters models are working but EXTREMELY SLOW.

For this demonstration, we'll use Termux, an amazing app available for both rooted and non-rooted Android users, as our foundation.

Once Termux is installed on the Xiaomi 13 Ultra, we'll install and run Ubuntu 20.04.

Following that, we'll install Ollama, a freely available open-source software, from Ollama.com

AI Installation on your smartphone Step by Step:​

1. Install Termux from Google Play or GitHub or F-Droid

2. Install Ubuntu 20.04 on Termux using this step-by-step guide: https://github.com/tuanpham-dev/termux-ubuntu

On Termux execute the following command in order to install a console-only version of Ubuntu 20:

NOTE: For all commands bellow the easiest way is to open this HOWTO on the browser on your phone, copy the commands from the HOWTO and paste them on the console.

pkg install wget curl proot tar -y && wget https://raw.githubusercontent.com/tuanpham-dev/termux-ubuntu/master/ubuntu.sh && chmod +x ubuntu.sh && bash ubuntu.sh nde

During the Ubuntu installation you have to answer few times with Y on the questions on the console.

Just before the end you have to add user and password for your user, also u have to enter a root password.

After each input hit Enter on the keyboard to continue.

After the end of the Ubuntu 20 installation you will be logged in automatically in Ubuntu.

For the NEXT time in order to start the Ubuntu OS you have to run this command on the Termux console:

./start-ubuntu20.sh

On the first start of the system run also this commands:

sudo apt update
sudo apt upgrade
sudo apt install curl

On all questions on the console answer with Y ... or just read what it's asking u :)

At the moment of writhing of this HOWTO u can Install Ollama by executing this command:

curl -fsSL https://ollama.com/install.sh | sh

It's highly advisable to visit the Ollama.com web site and follow the Linux installation instructions.

Start the Ollama service with this command:

ollama serve > /dev/null 2>&1 &

Download on your smartphone and run the desired LLM.​

For this guide, we'll install and test this LLM:

Mistral:7.3B - a 7.3 Billion parameters LLM:

Ensure you have 4.1 GB of space on your memory card.

The first execution of the following command will downloads the LLM.

Subsequent executions run the already downloaded LLM:

ollama run mistral:latest

The download time depends on your internet connection speed.

Once downloaded, the command automatically starts the LLM, presenting a console prompt for your queries.

>>>

Ollama prompt options:​

Ending a session with the chatbot is as simple as typing /bye at the prompt.

For a full list of available prompt parameters, type /help.

To completely close the Ollama session, enter /bye,

and then exit your Ubuntu session in Termux with:

exit

That's all, folks!

You're now equipped to explore the vast universe of LLMs right from your smartphone.

Happy experimenting, and may your queries always lead to enlightening answers.

Good luck!

Lupo

I have recorded 2 videos on running Mistral:7B and Llama2:7B LLM on Xiaomi 13 Ultra and Redmi Note 10 Pro which are available bellow:

Ollama LLM Mistral:7B and Llama2:7b run on Xiaomi 13 Ultra 12GB RAM​

Ollama LLM Mistral:7B and Gemma:2b run on Redmi Note 10 Pro 6GB RAM​

​

P.S Yoda the Hacker wisdom by Mistral 7B LLM AI​

Lupo Prompt: Tell me some Yoda the hacker wisdom

Mistral 7B LLM: Yoda, the wise Jedi Master from Star Wars, might not have been a hacker in the modern sense, but he certainly had valuable wisdom to share regarding efficiency, problem-solving, and adaptation.

Here are some Yoda-inspired "hacker" quotes:

1. "Luminous beings are we, not this crude matter." (Embrace the intangible nature of code and data.)

2. "Do or do not. There is no try." (Stay focused and persistent in your hacking endeavors.)

3. "Patience you must have, my young padawan." (Be patient while debugging issues and learning new skills.)

4. "Fear is the path to the dark side. Fear leads to anger. Anger leads to hate. Hate leads to suffering." (Avoid fear in your hacking journey and maintain a positive mindset.)

5. "Always in motion is the future." (Adapt and evolve with the ever-changing landscape of technology and hacking.)

6. "Much to learn, you still have." (Stay curious and continue learning new skills and techniques.)

7. "Size matters not." (Regardless of your age, experience, or resources, you can make a difference in the world of hacking.)

8. "Hmmm… A good question. You must ask yourself what does it mean to truly 'hack'?" (Reflect on the ethical implications and motivations behind hacking.)

Lupo :)