Edit model card

Model Card

This is my first fine tuned LLM project.

Usage

from transformers import GPT2LMHeadModel, GPT2Tokenizer

finetunedGPT = GPT2LMHeadModel.from_pretrained("rakeshkiriyath/gpt2Medium_text_to_sql")
finetunedTokenizer = GPT2Tokenizer.from_pretrained("rakeshkiriyath/gpt2Medium_text_to_sql")

def generate_text_to_sql(query, model, tokenizer, max_length=256):
    prompt = f"Translate the following English question to SQL: {query}"

    input_tensor = tokenizer.encode(prompt, return_tensors='pt').to('cuda')

    output = model.generate(input_tensor, max_length=max_length, num_return_sequences=1, pad_token_id=tokenizer.eos_token_id)

    decoded_output = tokenizer.decode(output[0], skip_special_tokens=True)

    # Return only the SQL part (removing the input text)
    sql_output = decoded_output[len(prompt):].strip()

    return sql_output

queryList = ["I need a list of employees who joined in the company last 6 months with a salary hike of 30% ",
             "Give me loginid,status,company of a user who is mapped to the organization XYZ "]

for query in queryList:

  sql_result = generate_text_to_sql(query, finetunedGPT, finetunedTokenizer)
  print(sql_result,"\n")

Output

SELECT COUNT(*) FROM employees WHERE last_6_months = "6 months" AND salary_hike = "30%"
SELECT loginid,status,company FROM user_mapped_to_organization WHERE mapping = "XYZ"

Training Hyperparameters

num_train_epochs=1
per_device_train_batch_size=3
gradient_accumulation_steps=9
learning_rate=5e-5
weight_decay=0.01

Evaluation

Step Training Loss
500 0.337800
1000 0.262900
1500 0.253200
2000 0.246400

{'eval_loss': 0.23689331114292145, 'eval_runtime': 104.4102, 'eval_samples_per_second': 67.043, 'eval_steps_per_second': 8.38, 'epoch': 1.0}

Downloads last month
301
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train rakeshkiriyath/gpt2Medium_text_to_sql