K8sAIOps/kubernetes_operator_1b_peft_gguf

kubectl Operator Model

• Developed by: dereklck
• License: Apache-2.0
• Fine-tuned from model: unsloth/Llama-3.2-1B-Instruct-bnb-4bit
• Model type: GGUF (compatible with Ollama)
• Language: English

This Llama-based model was fine-tuned to generate kubectl commands based on user descriptions. It was trained 2x faster with Unsloth and Hugging Face's TRL library.

---

Model Details

Purpose

The model assists users by:

• Generating accurate kubectl commands based on natural language descriptions.
• Providing brief explanations about Kubernetes for general queries.
• Requesting additional information if the instruction is incomplete or ambiguous.

Intended Users

• Kubernetes administrators
• DevOps engineers
• Developers working with Kubernetes clusters

Training Process

• Base Model: Unsloth's Llama-3.2-1B-Instruct-bnb-4bit
• Fine-tuning: Leveraged the Unsloth framework and Hugging Face's TRL library for efficient training.
• Training Data: Customized datasets focused on Kubernetes operations and kubectl command usage.

Performance

While the model aims to generate accurate commands, there might be occasional hallucinations or inaccuracies. Users should:

• Validate the generated commands before execution.
• Use the model as a guide rather than an absolute source.

---

Usage

Prompt Template

The model uses the following prompt template to generate responses:

You are an AI assistant that generates kubectl commands based on the user's description. If the instruction is incomplete or ambiguous, politely ask for more information. For general questions about Kubernetes, provide a brief explanation.

### Instruction:
<Your instruction here>

### Response:
<Assistant's response>
</s>

Parameters

• Temperature: 0.3 (controls the randomness; lower values make outputs more deterministic)
• Stop Token: </s> (indicates the end of the assistant's response)

---

Examples

Example 1: Generating a Command

Instruction:

List all pods in the kube-system namespace.

Response:

kubectl get pods -n kube-system
</s>

---

Example 2: Handling Ambiguity

Instruction:

Restart the deployment.

Response:

Could you please specify the name of the deployment you want to restart?
</s>

---

Example 3: Providing Explanations

Instruction:

What is a Kubernetes service?

Response:

A Kubernetes Service is an abstraction that defines a logical set of Pods and a policy by which to access them, often used to expose applications running on a set of Pods to other services or end-users.
</s>

---

Deployment with Ollama

Prerequisites

• Install Ollama on your system.
• Ensure you have the GGUF model file (kubectl_operator.Q8_0.gguf).

Steps

1. Create the Modelfile

   Save the following content as a file named modelfile:

   FROM kubectl_operator.Q8_0.gguf
   
   SYSTEM "You are an AI assistant that generates kubectl commands based on the user's description. If the instruction is incomplete or ambiguous, politely ask for more information. For general questions about Kubernetes, provide a brief explanation."
   
   PARAMETER temperature 0.3
   PARAMETER stop </s>
   
   TEMPLATE """
   You are an AI assistant that generates kubectl commands based on the user's description. If the instruction is incomplete or ambiguous, politely ask for more information. For general questions about Kubernetes, provide a brief explanation.
   
   ### Instruction:
   {{ .Prompt }}
   
   ### Response:
   {{ .Response }}
   </s>
   """
   

2. Create the Model with Ollama

   Open your terminal and run the following command to create the model:

   ollama create kubectl_operator -f modelfile
   

   This command tells Ollama to create a new model named kubectl_operator using the configuration specified in modelfile.

3. Run the Model

   Start interacting with your model:

   ollama run kubectl_operator
   

   This will initiate the model and prompt you for input based on the template provided.

---

Limitations and Considerations

• Accuracy: The model may occasionally produce incorrect or suboptimal commands. Always review the output before executing.
• Hallucinations: In rare cases, the model might generate irrelevant information. If the response seems off-topic, consider rephrasing your instruction.
• Security: Be cautious when executing generated commands, especially in production environments.

---

Feedback and Contributions

We welcome any comments or participation to improve the model and dataset. If you encounter issues or have suggestions for improvement:

• GitHub: Unsloth Repository
• Contact: Reach out to the developer, dereklck, for further assistance.

---