Run Command-R on CPU

#19
by J22 - opened

ChatLLM.cpp now support Command-R:

./bin/main -i -m ../c4ai_command_r_q4_0.bin
    ________          __  __    __    __  ___ 
   / ____/ /_  ____ _/ /_/ /   / /   /  |/  /_________  ____  
  / /   / __ \/ __ `/ __/ /   / /   / /|_/ // ___/ __ \/ __ \ 
 / /___/ / / / /_/ / /_/ /___/ /___/ /  / // /__/ /_/ / /_/ / 
 \____/_/ /_/\__,_/\__/_____/_____/_/  /_(_)___/ .___/ .___/  
You are served by Command-R,                  /_/   /_/       
with 34980831232 (35.0B) parameters.

You  > write a quick sort function in python
A.I. > Sure! Here's a quick sort function in Python:

def quick_sort(arr):
    if len(arr) <= 1:
        return arr

    pivot = arr[len(arr) // 2]
    left = [x for x in arr if x < pivot]
    middle = [x for x in arr if x == pivot]
    right = [x for x in arr if x > pivot]

    return quick_sort(left) + middle + quick_sort(right)

This function takes a list as input and returns a sorted list using the ....

Cool man!!!

how many ram i need to run it with cpu ?

For 8k context window (default), minimum 40GB for Q4 quantization.

Cohere For AI org

This is great @J22 -- going to close this for now but really nice addition.

sarahooker changed discussion status to closed

Sign up or log in to comment