README

README.md

@@ -10,4 +10,44 @@ pinned: false
 license: mit
 ---
 
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+# Dataset
+
+The dataset usesd is https://huggingface.co/datasets/lmsys/chatbot_arena_conversations 
+
+Preprocessing:
+- filtered german conversations
+- took first user prompt
+- deleted short prompts (less than 70 chars)
+
+```python
+dataset = load_dataset('lmsys/chatbot_arena_conversations')
+
+def get_message(x):
+    x['message'] = [x['conversation_a'][0]]
+    return x
+
+dataset = dataset.filter(lambda x: x['language'] == 'German')
+dataset = dataset['train'].map(get_message)
+dataset = dataset.filter(lambda x: len(x['message'][0]['content']) > 70)
+```
+
+# Generation
+
+I rely on the huggingface `conversational` pipeline to generate the outputs. There are some issues with the chat template (esp. for the non-instruction tuned models) i'll fix later.
+
+```python
+messages = json.loads(Path('messages.json').read_text())
+outputs = []
+pipe = pipeline(
+    "conversational",
+    model=model_name,
+    torch_dtype="auto",
+    device_map=device,
+    max_new_tokens=1024,
+    trust_remote_code=True
+)
+
+for message in tqdm(messages):
+    output = pipe([message])
+    outputs.append(output)
+```

app.py

@@ -26,7 +26,7 @@ with gr.Blocks() as iface:
         drop_model1 = gr.Dropdown(models, label='Model 1', value=random.choice(models))
         drop_model2 = gr.Dropdown(models, label='Model 2', value=random.choice(models))
     with gr.Row():
-        btn = gr.Button("Run")
+        btn = gr.Button("Show Outputs")
     with gr.Row():
         out_message = gr.TextArea(label='Prompt')
     with gr.Row():