utkarshshukla2912 commited on
Commit
fb71843
·
verified ·
1 Parent(s): d626b35

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +24 -0
README.md CHANGED
@@ -24,6 +24,30 @@ You can plug it into your calling or voice AI stack to automatically extract:
24
 
25
  It’s built to handle real-world Hindi, English, and mixed Hinglish calls, including noisy transcripts.
26
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
 
28
  - **Developed by:** RinggAI
29
  - **License:** apache-2.0
 
24
 
25
  It’s built to handle real-world Hindi, English, and mixed Hinglish calls, including noisy transcripts.
26
 
27
+ Training Parameters:
28
+ ```
29
+ rank = 64
30
+ lora_alpha = rank*2,
31
+ target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
32
+ "gate_proj", "up_proj", "down_proj",],
33
+ SFTConfig(
34
+ dataset_text_field = "prompt",
35
+ per_device_train_batch_size = 32,
36
+ gradient_accumulation_steps = 1, # Use GA to mimic batch size!
37
+ warmup_steps = 5,
38
+ num_train_epochs = 3,
39
+ learning_rate = 2e-4,
40
+ logging_steps = 50,
41
+ optim = "adamw_8bit",
42
+ weight_decay = 0.001,
43
+ lr_scheduler_type = "linear",
44
+ seed = SEED,
45
+ report_to = "wandb",
46
+ eval_strategy="steps",
47
+ eval_steps=200,
48
+ )
49
+ The model was finetuned on ~100,000 curated transcripts across different domanins and language preferences
50
+ ```
51
 
52
  - **Developed by:** RinggAI
53
  - **License:** apache-2.0